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ABSTRACT 


The purpose of this study was to determine whether Naval Medieine’s current 
Information Assurance Policy and resultant efforts properly address federal requirements or 
current threats confronting Naval Medicine information technology professionals. 

The primary research was conducted with a survey instrument detailing thirty 
questions with various response categories. The findings of the survey questionnaire 
revealed the existing numbers of previously compromised systems were directly related to 
the frequency of vulnerability scanning and remediation practices in the current threat 
environment. 

This study will provide insight to anyone interested in the future assessment of 
Naval Medicine’s information security posture. These findings have important implications 
for command personnel charged with the responsibility and accountability of Naval 
Medicine’s networks and data systems, as well as other communities throughout the Navy. 
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EXECUTIVE SUMMARY 


The numbers of well-known software vulnerabilities eontinue to inerease and will 
likely worsen as even more powerful and eomplex applieations emerge. Traditionally, 
information teehnology managers have eombated network vulnerabilities with a variety of 
approaehes. The most eornmon involves applying the latest pateh (“hot fix”) or serviee 
paek to a eomputer system, while maintaining stringent aeeess eontrol lists (ACLs) on 
routers and firewalls to filter network traffie. This last method, though somewhat effeetive, 
is limited in that even the best effort approaches to separating good and bad network traffic 
can be circumvented. Unfortunately, technologies such as these are intended to serve 
primarily as perimeter network defense systems, but have quickly become the panacea of 
network security. In many cases, a false sense of security has been created and a large 
number of networks fall prey to well-known exploits because the recommended system 
patches have not been applied to vulnerable systems. Additionally, 80% of system 
compromises originate within the local network, leaving the perimeter controls at a 
significant disadvantage (Konigsberg, 2002). While the most basic tenets of securing 
computing systems are the application of system patches, fewer patches are being applied as 
networking environments become more complex. 

The proposed research within this thesis evaluated whether automated network 
vulnerability scanning software solutions would provide a reliable and cost effective means 
to manage the growing numbers of operating systems and applications vulnerabilities, while 
providing a greater ability to comply with federal requirements in the area of information 
security practices for Naval Medicine components. This project also provided an analysis of 
the total number of systems compromises over the past 12 months and concluded that 
vulnerability scanning and remediation procedures were not being performed expediently 
enough to meet the current information assurance threats. 

Knowing there will always be differences in the way organizations respond to 
potential threats, common to them will be maintaining an effective patch management 
program, becoming even more important as zero-day exploits begin to appear on a more 
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regular basis. The ability to ineorporate this, as well as new and emerging eoneepts and 
praetiees, will ultimately determine the future sueeess of any organization’s information 
seeurity program. 

It should be noted that there are other seeurity models that do not depend on pateh 
management, or supplement it, but will not be eovered in this work. Managing and patehing 
systems is now a way of life in our industry, and until sueh time as they are unneeessary, 
they need to be systematieally ineorporated into our polieies. 
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L INTRODUCTION 

A, SITUATION ANALYSIS 

The number of well-known software vulnerabilities, on average, has doubled 
every year since 1998 (CERT, 2003) and will likely worsen as even more powerful and 
complex applications emerge. In 2002, more than 4,000 well known vulnerabilities were 
listed and over 82,000 incidents were reported to CERT (CERT, 2003). Surprisingly, 
99% of reported incidents resulted from not having well-known exploits patches on 
affected computing systems (Shipley, January 23, 2003). 

Traditionally, information technology professionals have thwarted network 
vulnerabilities using a variety of approaches. One of the most common involves applying 
the latest patch or service pack to a computer system, while maintaining stringent access 
control lists (ACEs) on routers and firewalls to filter network traffic. It is well 
recognized that even the best approaches to separating good and bad network traffic can 
be circumvented, which has lead to numerous implementations of Intrusion Detection 
Systems (IDSs) to provide notification of suspected malicious network traffic. 

Modern approaches to network security are focused on signature based 
recognition and access control lists (ACEs), such as are found in firewalls and routers, 
and Intrusion Detection System (IDS) monitoring. Unfortunately, these technologies are 
intended to serve primarily as perimeter network defense systems, but have quickly 
become the perceived panacea of network security. In many cases, because the 
recommended system patches are not applied to vulnerable systems, a large number of 
networks fall prey to a false sense of security from the aforementioned perimeter defense, 
and are victims of well-known exploits. Additionally, 80% of system compromises 
originate within the local network, leaving the firewall and certain IDS at a significant 
disadvantage (Konigsberg, 2002). Recent surveys also indicate that the majority of 
attacks are directed at port 80, which has traditionally not been filtered since it facilitates 
Web traffic (Burns, 2003). While the most basic tenets of securing computing systems 
are the application of system patches, it has become a seemingly less practiced task as 
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networking environments beeome more eomplex, and defenses are being thrust to the 
perimeter. 

Maintaining a relatively secure computing network has become a comprehensive 
task for many information technology managers. Preparing for the next wave of system 
exploits to approach the Internet remains a mystery to many information managers unable 
to keep abreast of the trends. Commonly known vulnerabilities and the attacks associated 
with them are well documented, such as the buffer overflow; however, these 
vulnerabilities have not been fully addressed and corrected by the vendors for a variety of 
reasons. For those that even realize the importance of patching systems, many have 
concerns of system patch incompatibility and fear that the available patches may disrupt 
or negatively impact the computing system operation. Protecting information systems 
can equate to ensuring that the most recent software patches have been applied to every 
known vulnerable system, but knowing of every patch or update, and their possible side- 
effects, becomes a virtually impossible task. Management of these systems becomes a 
somewhat daunting task then, as the number of vulnerabilities increases, the number of 
systems to be managed grows, and information technology staffing remains the same or 
decreases due to cutbacks. 

The proposed research evaluates whether automated network vulnerability 
scanning software solutions can provide a reliable and cost effective means to manage the 
growing numbers of operating systems and applications vulnerabilities, while providing a 
greater ability to comply with federal requirements in the area of information security 
practices. The research focuses on determining what, if any, formal patch management 
practices exist and how current actions can be supplemented with automated vulnerability 
scanning and patching technologies. 

This study is of particular importance to the command personnel charged with the 
responsibility and accountability of Naval Medicine’s networks and electronic data 
systems. The supervisors, educators and trainers of today will develop the leaders of 
tomorrow, who will become responsible for ensuring that mission essential objectives are 
completed. To accomplish this, leaders must know what people need or desire to get the 
best performance from them. This research offers practical information regarding 
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modem patch management techniques and the technologies available to assist them in 
that effort. 

The Internet has become essential for most organizations and has grown 
exponentially in the number of private parties obtaining access each year. As more 
people engage in electronic activity, the potential threat increases at the same rate (Azari, 
2003). There will always be differences in the way people and organizations respond to 
potential threats, but maintaining an effective patch management program will become 
even more important as zero-day exploits begin to appear. The ability to incorporate this, 
as well as new and emerging concepts and practices will ultimately determine the future 
success of any organization’s information security program. 

B, PREMISE AND HYPOTHESIS 

Based on prior experience handling incident reports of subordinate command 
computer compromises, and following through with mitigation of known vulnerabilities 
within Naval Medicine, observation suggests at least three out of four compromises 
resulted from lack of timely patch administration. In conducting the research for this 
thesis, this observational figure posed a suitable point from which to pursue the following 
hypothesis: 75 percent of Naval Medicine’s known information systems compromises 
were not protected by the available vulnerability patch(es). 

C. DEFINITIONS 

1. Naval Medicine 

The Department of the Navy healthcare organization, composed of approximately 
400 individual units responsible for maintaining the health of all Navy and Marine Corps 
personnel. 

2. Information System 

Hardware and software, application programs and devices that input, process, 
store and/or output electronic data elements. 

3. System Compromise 

Any unauthorized system events or data theft occurring on an information system. 

4. Vulnerability Patch 
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A software and or hardware vendor remedy that eorreets known system 
vulnerabilities or operating system errors. 

D, DISCLAIMER AND LIMITATIONS 

The information used to eompile researeh findings is utilized to satisfy the 
aeademie reporting requirements needed for eompletion of a Master of Seience degree in 
Information Systems Technology from the Naval Postgraduate School. Surveys will be 
limited in distribution to Naval Medicine Chief Information Officers. Additionally, 
completed survey questionnaires and any other organizational data utilized within this 
research paper will be held in strict confidence and used solely for the indicated purpose. 

Secondary research efforts may be limited by the number of personnel employed 
within Naval Medicine available, or their willingness to participate in the Information 
Assurance Management Survey. The total percentage of known information systems 
compromises that were not protected by the available vulnerability patch will be 
extracted from the survey results to validate or invalidate the premise. Additionally, the 
length of the academic term further limits the scope of the study. Lastly, the efforts of 
this research may not accurately represent the other components of the United States 
Navy in regards to Chief Information Officers or any other Department of Defense 
representatives fitting the above mentioned titles. 
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11. BACKGROUND 


A, INTRODUCTION 

Today, the only thing that seems eertain is ehange. Business and operational 
environments have undergone significant transformations over the past decade. The 
constant stream of differing and more powerful Internet technologies, networked systems, 
and applications have refined processes and simultaneously created a surfeit of system 
vulnerabilities. This problem tends to increase each time a new technology or a system 
with more features is introduced. If the increasing complexity is coupled with other 
issues associated with expanding enterprises, it is relatively easy to imagine that the 
proper management and control of such technologies could be difficult at best. A 
primary concern regarding these technologies is ensuring information security; though it 
is often one of the most difficult items to justify in annual budgets as it becomes more 
difficult to assess, prioritize and measure corrective methods to counter the known risks 
or threats. Personnel, knowledge, funding, tools, and training are therefore seemingly 
obvious deficiencies in information security environments. 

B, CURRENT GLOBAL NETWORK THREATS 

A small nation-state now has the ability to cripple a large adversary by 
compromising unprotected information system controls. Electrical grids, water treatment 
facilities, airline communications systems, financial systems, and many others are 
susceptible to compromises by anyone with the appropriate skill levels, equipment, and 
time. A system can be monitored and maintained to resist or repel known threats, but one 
sophisticated hacker can wreak havoc in minutes. A well-publicized example of how 
quickly and successfully an attack can be performed was demonstrated by the Slammer 
Worm (AKA: Sapphire), which was released on 25 January 2003. The Slammer worm 
was the fastest-spreading worm in computing history, primarily due to its small total size 
of 404 bytes, which included the header. Slammer was an exploit of a buffer overflow in 
Microsoft’s SQL server and applications created with the Microsoft Server 2000 Desktop 
Engine. Within 3 minutes of its release, the total number of infected hosts doubled every 
8.5 seconds. Thirty minutes later, the worm and its clones were scanning 55 million IP 
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addresses per second. Within 40 minutes, approximately 90% of all susceptible hosts had 
been compromised (McGuirl, 2004). 

Although Slammer was not carrying a malicious payload, it did cause significant 
collateral damage. Twenty-seven million Korean mobile phone and Internet accounts 
were offline, more than a 100,000 Portuguese cable modems were offline, 13,000 Ra nk 
of America ATMS were downed, and emergency service providers in Seattle lost 
dispatch capabilities for hours while attempting to service a community of 700,000 
people. Mi2g Limited, an English security firm, estimated that Slammer costs reached 
$1.2B in productivity losses (McGuirl, 2004). 

The Slammer incident is one of many examples where a vendor patch was 
available for a well-known exploit for more than 6 months, but had not been applied to 
the affected systems. Slammer is not the first, nor the last, to be seen. We must not 
forget while history repeats itself, in cyberspace it replicates. Two previous worms that 
caused similar issues were the Code Red IIS ISAPI buffer overflow attack and the Nimda 
Worm that exploited an IIS Web traversal vulnerability. Again, anyone who experienced 
these attacks could have prevented them if they had simply applied the patch 3 to 4 weeks 
after the vulnerabilities had been announced. 

Slammer originated with one initial instance of a compromise. If one considered 
that a number of these exploit attempts and break-ins occur on a daily basis, the concept 
may impose more concern. To determine just how much malicious activity occurs on the 
Internet, I-trap Security Services, based in Cleveland, Ohio, monitored and analyzed two 
weeks of internet traffic from a 10,000 node ISP enterprise that serviced Tel Aviv 
University, the largest university in Israel. A two-week sampling recorded 180,000 
attack events. Those events consisted of scanning and actual break-in attempts. 
Approximately 96 percent of the recorded scans were followed by attacks from the same 
source. That is a staggering number in itself, but it is more important to recognize that 
roughly 90% of those attacks are generated by worm activity. Any organization relying 
on perimeter controls such as firewalls, router access control lists, intrusion prevention 
systems or anti-virus tools would not have been protected. The I-trap report indicated 
that most of the attacks originating from China and the United States were automated; 
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however, it should be noted that attaek totals were supported by 99 differing countries 
around the globe. Another interesting note from the sample was that more than 139,000 
of those attacks (75%) were directed at port 80, the port used for standard Web (HTML) 
page transfers. (Bums, 2003) 

Today’s most competitive organizations, whether private, corporate or 
government-funded, are employing their personnel with the fastest and most efficient 
computing systems to perform their tasks. The majority of those systems interface with 
the Internet, and the electronic transactions and/or data stored on those systems is at risk 
- some more than others, but at risk nonetheless. If these risks are discovered by anyone 
with malicious intent, the organization’s image and livelihood can be damaged in a 
number of ways. System compromises can result in, but are not limited to, media 
attention, public embarrassment, financial losses, whether stolen or incurred by penalty, 
increased maintenance for restoration efforts, and loss of productivity in downed systems. 
Protecting information system assets in a globally connected world requires a dedicated 
amount of time and funding to counter the existing threats. A near real time vulnerability 
scanning and patching process is the last line of defense in protecting information assets 
once perimeter filters have been breached (Nicolett and Pescatore, 2003). 

C. CULTURE CONTRIBUTORS 

Computing systems technology has reached approximately 25 percent of homes 
worldwide in the past decade (Azari, 2003). The world we lived in just ten years ago has 
been replaced with much more convenient and less expensive methods of performing 
daily tasks, methods driven by technology that affects nearly everything imaginable in 
our daily lives. These changes range from daily transactions involving purchases and 
sales to monetary transfers, safety devices, and communications systems, among many 
others. Global connections link the majority of markets and institutions around the world 
and significantly affect the overall economy. Military components tout cyber warfare as 
the new order for combat operations, and a number of technology-driven weapons 
systems, often referred to as smart weapons, are fast replacing conventional methods of 
warfare. 
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The technological revolution even affects those millions that don’t use the 
Internet or other technological advances. As previously mentioned, anyone relying on 
electricity, purified water, public transportation or financial systems can be affected by 
technology because it also drives the majority of those systems. This continued reliance 
on technology will continue to enforce its utilization and dependence. 

D. VULNERABILITY STATISTICS 

Although it was not the first, one of the most widely recognized worms was 
designed by Robert Morris in 1998. The Morris worm introduced many to the reality of 
cyber threats as it invaded approximately 6000 computers within a couple of hours. In 
1998, this figure represented 10% of the entire Internet. The worm was not destructive, 
but it did prove the powers of a buffer overflow. This event had two beneficial 
outcomes: the realization that dangers do exist in a connected world, and more 
importantly, the genesis of the Computer Emergency Response Team (CERT), which 
was developed as a notification and dissemination point for known vulnerabilities. The 
CERT Coordination Center (CERT/CC) is supported by federal research funds and is 
operated by Carnegie Mellon University. Many information assurance practitioners have 
reviewed their advisories about vulnerabilities, bugs, patches, and where to find the fixes 
to those known problems (Rubin, 2001). Eigures 1 through 3 display the number of 
reported incidents, vulnerabilities and the percentages of increase from 1988 through 
2003, respectively. 

The number of incidents reported over the past five years produces a wide range 
of assumptions. The Internet, and incidence of malicious code, has grown substantially. 
Reported incidents may not necessarily be the actual number of incidents as many 
organizational reputations may be at risk for simply admitting they have experienced an 
incident. What may not be as obvious is the fact that many more organizations are 
reporting incidents because they are more familiar with the occurrence of system 
compromises. 
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Figure 1. Number of Ineidents Reported. From CERT/CC Statistics For Incidents (2003). 
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Figure 2. Number of Vulnerabilities Reported. From CERT/CC Statistics For 

Vulnerabilities (2003). 
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The number of new vulnerabilities speeifieally indicates those that have not been 
seen before. Many are reiterations of earlier ones, but they are still different. On 24 
February 2004, at the annual RSA panel discussion, Quals reported that “the lifespan of 
some vulnerabilities is unlimited” and that “50% of the most prevalent and critical 
vulnerabilities are being replaced on an annual basis”(Eschelbeck, 2004). Quals also 
presented a differing total number of vulnerabilities based on vulnerability data from 
December 2003. Their presentation submitted that there have been a total 3,011,000 IP 
scans, 1,905,000 total critical vulnerabilities, 2,054 different vulnerabilities and 1,175 
different critical vulnerabilities. Their definition of critical was defined as “Providing an 
attacker the ability to gain full control of the system and/or leakage of highly sensitive 
information. For example, vulnerabilities may enable full read and/or write access to 
files, remote execution of command, and the presence of backdoors”(Eschelbeck, 2004). 

During 2002, the Security Alert Consensus said there were approximately 1000 
new operating system and applications vulnerabilities, which equates to roughly 83 new 
vulnerabilities per month. During 2003, SecurityEocus reported 7,679 vulnerabilities in 
their database, while NISTS ICAT metabase listed only 5,712 and the Common 
Vulnerabilities and Exposures Group at mitre.org listed only 2,573 (Shipley, June 26, 
2003). It seems rather confusing that such drastic differences are reported, but the 
important thing to remember is that there are thousands of vulnerabilities that have been 
identified and there are likely thousands more that have not been discovered. Becoming 
aware of them and guarding information systems from those threats are the only ways a 
connected organization will be able survive the onslaught of malicious code floating 
around in cyberspace. “In short, when it comes to compromise of data confidentiality, 
what you don’t know can really hurt you” (Rubin, 2001). 
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As of 20 May 2004, the past four years of NIST-analyzed vulnerability types were 
as follows: 


Statistics on all NIST Analyzed Vulnerabilities 

Vulnerability Type 

2004 

2003 

2002 

2001 

Vulnerability Count 

264 

1007 

1307 

1506 

Remote Attack 

193 (73%) 

755 (75%) 

1052 (80%) 

1056 (70%) 

Local Attack 

75 (28%) 

252 (25%) 

275 (21%) 

524 (35%) 

Denial of Service 

68 (26%) 

281 (28%) 

330 (25%) 

419 (28%) 

OS Vulnerabilities 

56 (21%) 

163 (16%) 

212(16%) 

248 (16%) 


Table 1. Statistics on all NIST Analyzed Vulnerabilities. 
From http://icat.nist.gov/vt portal.cfm 


Note: This table shows the distribution of various vulnerability characteristics. The raw 
number in each cell is the number of vulnerabilities that meet that particular 
characteristic for that year. The percentage to the right of each raw number is the 
percentage of vulnerabilities having that particular characteristic for that year. 

The increases and a select number of decreases in vulnerabilities and incidents 
can be directly related to the widespread releases of new operating systems software and 
applications, while additional decreases can be attributed to the investments made in 
information assurance practices to combat such threats. Some analysts contend that the 
rise in reports is due to the increased numbers of people that are monitoring the network 
and the recent expansions of the unsophisticated consumer market obtaining broadband 
connections such as DSL and cable modems. These types of connections are always on 
and present immediate dangers to the system owner if not configured or protected by 
filtering devices such as firewalls. (Goth, 2004) 

E. THE THREAT PERSPECTIVE 

The United States military services are not excluded from the same vulnerabilities 
that confront the rest of the free world. To get an idea of just how many times the 
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military services have faced vulnerability incidents, consider the following links that 
showcase past hacker activities. 

Current attack postings are available at zone-h.org at http://www.zone- 
h.org/en/defacements/ . It is disturbing to know that new attacks are posted nearly every 
minute of every day. At certain times, there are multiple attacks occurring within one 
minute around the globe. This site clearly displays the operating system that was 
compromised. A quick review will provide all the proof required to dispel the myth that 
some operating systems cannot be breached. 

The digital attacks archive link on the left column of zone-h.org’s web page will 
redirect a Web browser to the primary archive. If desired, add the filter “.mil” to view 
what the services have encountered in the past. OSD, SPAWAR, and even some of Navy 
Medicine's Web pages are easily found within the archive. A visit to the breakout li nk 
within the attrition.org site at http://www.attrition.org/mirror/attrition/ allows one to view 
even more .mil and other federal agency defacements. According to this mirror site, 
since July 1999, 186 defacements have occurred on .mil domains, and 42 (approximately 
23%) of those were Navy-specific. 

F. THREAT AWARENESS AND RESPONSE RESOURCES 

A significant number of resources exist to alert and help information systems 
personnel defend their assets. Many of them are listed on the NIST Vulnerability and 
Threat Portal http://icat.nist.gov/vt portal.cfm Links to other competent vulnerability 
notification organizations such as The US Computer Emergency Readiness Team (US- 
CERT), the Carnegie Mellon Software Engineering Institute, the National Institute of 
Standards and Technology, and the SANS Institute are linked from the NIST portal. 
Each of them differs slightly, but each is an excellent resource. Users can submit to 
mailing lists for frequent vulnerability updates. 

The US Department of Homeland Security (DHS) employs the US Computer 
Emergency Readiness Team (US-CERT) and provides the US-CERT Current Activity 
Web page, which offers an up to date summary of the most frequent and devastating 
types of information security incidents. This resource is located at: http://www.us- 
cert.gov/current/current activitv.html . The CERT/CC Incident Notes Web page is 
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maintained by the Carnegie Mellon Software Engineering Institute and is located at 
http://www.cert.org/incident_notes/. Another feature page maintained by the Carnegie 
Mellon CERT® Coordination Center provides Steps for Recovering from a UNIX or NT 
System Compromise. That resource can be found at http://www.cert.org/tech tips/win- 
UNIX-system compromise.html . 

The ICAT Metabase is maintained by the Computer Security Division at the 
National Institute of Standards and Technology. The ICAT is an index of searchable 
computer vulnerabilities. It also provides a search capability at a granular level that li nk s 
users to vulnerability and patch information and can be found at 
http://icat.nist.gov/icat.cfm . 

The System Administration, Networking, and Security Institute (SANS) 
showcases the SANS Top 20 Internet Security Vulnerabilities at 
http://www.sans.org/top20/ . The SANS Top 20 is the merger of two top-ten lists. 
Specifically, it provides the ten most commonly exploited vulnerable services in 
Windows and the ten most commonly exploited vulnerable services in UNIX and Einux 
operating systems. There are a great number of security incidents occurring every year 
that affect these popular operating systems, but the majority of the successful attacks 
focus on one or more of the twenty identified vulnerabilities. It should be noted that the 
twenty vulnerabilities are those that are considered by a number of security experts to be 
the most critical vulnerabilities that warrant immediate attention. The entire process is 
coordinated by leading security experts that practice security roles in some of the most 
information security-focused agencies around the world. This is not limited to, but 
includes information from, security vendors, consulting organizations and a number of 
the top university-driven security programs. The SANS Institute Internet site also 
maintains a reading room archive rich in resources pertaining to policy, risk assessment 
procedures as well as a number of other information security-related topics. 

East, but not least, Bugtraq is considered by many to be the most important 
Internet information security list. Vulnerabilities announcements are often posted here 
well in advance of the government-sponsored resources previously referenced. A number 
of current and previous archives can be found at http://www.ntbugtraq .com . 
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Each of these noted serviees is a dynamic and ever-changing resource, from step- 
by-step instructions to additional links regarding information useful for correcting known 
security flaws. Most generally, each of them provide feedback links for continuous 
improvement initiatives and encourage participation in fighting the good fight in the 
eontinuous battles involved in the information security arena. 

G. WHAT IT MAY TAKE TO COUNTER FUTURE THREATS 

The future holds many uncertainties, but the experts agree that the cyber universe 
is a dangerous place to conduet electronic transactions, whether business or personal. 
There are a number of reasons those dangers exist. The reality of the bits and bytes 
world is that nothing is bulletproof. NIST recently reported that 36% of vulnerabilities 
are resultant of eonfiguration or design problems, and the rest are due to programming 
errors. “Of those errors ‘the basic mistakes’—^buffer overflows, direetory traversal 
attaeks, format string vulnerabilities, symlink attaeks, cross-site seripting vulnerabilities, 
and shell metaeharacter issues—are responsible for 51 to 64 pereent of vulnerabilities” 
(Goth, 2004). Remediating the known vulnerabilities in a timely manner and configuring 
systems to repel attaeks remain the best known defenses. 

If these realities are not heeded, they will be costly in terms of lost data, downed 
systems, or legal penalties. To counter a threat, two basic concepts must be understood: 
The threat has to be identified; only then can it be responded to. Those two factors will 
determine your overall effeetiveness in thwarting the threat. NIST and other 
vulnerability summary organizations are now faeing the challenges of keeping pace with 
the outbreaks of vulnerabilities, identifying them so a response can be developed. The 
decreasing window of time between diseovery and remediation has ineited the need for 
even more efficient proeesses for determining the identifiers for inclusion into the 
industry standard Common Vulnerabilities and Exposures (CVE) Dietionary. The CVE 
in and of itself is not a database; it is simply a dictionary of vulnerabilities and exposures 
(Goth, 2004). 

To combat increasingly sophisticated and more aggressive eyber attaeks, 
everyone will be required to entertain new approaehes, tools and services to inerease their 
chance of survival. The standard praetices of waiting for alerts and patehing when 
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convenient is already becoming ineffective. It is predicted that “soon computers will faee 
‘Warhol’ threats that spread across the Internet and infect systems worldwide within 15 
minutes. In a few years, the Net will be hit by ‘flash threats that can spread in just 
seconds...’” (Evers, 2003). Leading researehers and seeurity experts further prediet that 
during 2004, the number of remote proeedure eall (RPC) exploits will continue to appear. 
The RPC is a primary eomponent required to manage elient-server eomputing. These 
proeedures are not restricted to Mierosoft operating systems and Jeff Moss, President and 
CEO of Blaek Hat, Inc. reported that haekers are now looking for areas that are not being 
addressed. “In particular, hackers are exploring ways to attack memory ‘heaps,’ or areas 
of computer memory that are created dynamically when programs run.” (Roberts, 2003) 

The ever-inereasing rates of information systems eompromises have also gained 
the immediate attention of the federal government over the past decade. As voters 
become more dissatisfied with the level of proteetion their private information is 
afforded, the requirements and penalties imposed for lack of due diligence will 
eontinually increase. The future of information assurance has already been addressed by 
a federal legislation. Sean Doherty published email poll results regarding the impact of 
the newest federal initiatives in the July 2003 edition of Network Computing magazine. 
The survey indicated that 72% of survey respondents were directly affected by the 
legislative mandates. “The Gramm-Leach-Bliley Aet (GLBA) and the Health Insuranee 
Portability and Accountability Act (HIPAA) hold affected enterprises aecountable to 
proteet private information, meaning IT must assess the risks and implement appropriate 
safeguards” (Doherty, 2003). Healtheare information losses resultant of ignoring the 
rules under HIPAA ean cost up to $25,000 per violation. Individual losses of personal 
information ean cost financial institutions $1,000 per individual or up to $500,000 for a 
elass of individuals who have not been afforded the appropriate protection mandated for 
them under the GLBA. 

Additionally, the Sarbane-Oxley Act of 2002 (SARBOX) mandates that any 
company issuing private securities maintain the appropriate eontrols of their financial 
reporting systems, as well as perform assessments of their systems’ eontrols and 
reporting those findings to the Securities and Exchange Commission (SEC). The 
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SARBOX legislation can impose fiscal penalties up to $1 million as well as ten years’ 
imprisonment for a corporate officer that knowingly endorses a false financial report. 

As a part of the Federal Information Security Act of 2002 (FISMA), Congress is 
requiring the National Institute of Standards and Technology (NIST) to develop guidance 
for IT management safeguards that will adequately address the information assurance 
security triad of confidentiality, integrity and availability of information systems and their 
data elements. NIST Special Publication 800-53 is expected to be finalized in the near 
future to detail required government entity controls by 2005, which will also include 
requirements for hardware and software maintenance. The 238-page draft version is 
currently available for comment. This document should be utilized in conjunction with 
two other NIST publications: the Federal Information Processing Standard Publication 
199: Standards for Security Categorization of Federal Information and Information 
Systems; and the NIST Special Publication 800-53, Recommended Security Controls for 
Federal Information Systems (Chabrow, 2003). 

The draft version of the National Strategy to Secure Cyberspace, September 2002, 
also highlights the changes required to properly address existing deficiencies in 
information assurance practices. On Tuesday, 3 December 2002 at the Computer System 
Security and Advisory Board Meeting, Richard Clarke, the Chair of the President’s 
Critical Infrastructure Protection Board “...indicated that the real problem was not the 
lack of threat analysis but of a vulnerability analysis.” (Clarke, 2002). According to the 
draft, a number of initiatives and practices will have to be adopted to protect information 
technology assets. Among those items that pertain to this project, section R3-5 in the 
summary section is most applicable. This section contends that “Federal agencies should 
continue to expand the use of automated, enterprise-wide security assessment and 
security policy enforcement tools and actively deploy threat management tools to 
preempt attacks” (Computer System Security and Privacy Advisory Board, 2002). 

If the information assurance initiatives to safeguard information assets can focus 
more on a holistic approach that addresses the security framework, specifically in the 
areas of policy, processes, personnel and products, a more recognizable sense of 
information security will be realized (McGuirl, 2004). 
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H, INFORMATION ASSURANCE POLICY REQUIREMENTS 

This section identifies the source of privacy and security requirements for Federal 
automated information systems with which DoD and Naval Medicine must comply. 
Governing Federal privacy and security policy statements express fundamental privacy 
and security requirements and serve as a framework for developing more specific 
technical and administrative security specifications, design, and operational requirements. 

1. Federal Requirements 

a. Privacy Act of1974, PL. 93-579, 5 U.S. C. 552a (1974) 

The Privacy Act requires federal agencies to safeguard personal data 
processed by automated information systems. This Act also requires the agencies to allow 
individuals to find out what personal information is being maintained and to correct 
inaccurate information. The Act identifies physical security procedures, information 
management practices, and computer and network controls for systems that process 
Privacy Act data. 

b. Computer Security Act of1987P.L. 100-235 (1988) 

The Computer Security Act, which went into effect in September 1988, 
requires every U.S. government computer system that processes sensitive information to 
have a customized security plan for the system's management and usage. It also requires 
all U.S. government employees, contractors, and others who directly affect federal 
programs undergo periodic training in computer security. All users of systems containing 
sensitive data must also receive computer security training corresponding to the 
sensitivity of the data to which they have access. 

c. The Clinger-Cohen Act of 1996 

The Clinger-Cohen Act requires all federal government agency heads to 
design and implement IT management processes for maximizing the value and assessing 
and managing the risks of the IT acquisitions. They are also directed to establish goals 
for improving the efficiency and effectiveness of agency operations through the effective 
use of IT. With regards to information assurance, they ensure that the information 
security policies, procedures, and practices of the agency are adequate. 
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d. Management of Federal Information Resources (0MB Circular 
No. A-130, Appendix III, "Security of Federal Automated 
Information Systems”, 1996 

0MB Circular No. A-130 Appendix III, revised in February 1996, stresses 
management eontrols, individual responsibility, aeeountability, and awareness and 
training, rather than teehnieal eontrols. Ageneies must ensure that risk-based rules of 
behavior and operation are established, that employees are trained in them, and that the 
rules are enforeed. Speeifieally, it requires ageneies to implement and maintain a 
program to ensure adequate seeurity is provided for all ageney information eolleeted, 
proeessed, transmitted, stored, or disseminated in general support systems and major 
applieations. Appendix III no longer requires a formal risk analysis. Instead, risk-based 
management is employed to address general risk assessments. Major risk-based 
management faetors inelude: applieations, threats, vulnerabilities, and safeguard 
effeetiveness. Lastly, eaeh ageney is required to work with 0MB, NIST, and NSA to 
improve ageney eomputer seeurity. 

e. Health Insurance Portability & Accountability Act of 1996, 
Public Law I04-I9I (HIPAA) 

The HIPAA Seeurity Rule speeifieally foeuses on the safeguarding of 
eleetronie proteeted health information (EPHI). The main goal of the HIPAA Seeurity 
Rule is to proteet the eonfidentiality, integrity, and availability of eleetronie proteeted 
health information. The Federal Information Seeurity Management Aet (FISMA) applies 
to all federal ageneies and all information types, but the HIPAA requirement further 
refines the rules for use of EPHI. All Naval Medieine faeilities and health eare providers 
must eomply with the HIPAA Seeurity Rule, whieh establishes a set of seeurity standards 
for seeuring eertain health eare information. A health eare provider is defined as any 
provider of medieal or other health serviees, or supplies, whieh transmits any health 
information in eleetronie form in eonneetion with a transaetion for whieh a standard has 
been adopted. 

f Presidential Decision Directive-63 (PDD-63), 1998 

This doeument reeognizes that the United States maintains the world's 
strongest military as well as the largest national eeonomy and that those aspeets of our 
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power are mutually reinforcing and dependent. It also recognizes that each aspect is 
increasingly reliant on certain critical infrastructures and cyber-based information 
systems. It further recognizes that although critical infrastructures had historically been 
physically and logically separate systems with little interdependence, they were 
increasingly dependent on information technology, and each other. The increased 
automation and links between them also created new vulnerabilities to equipment failure, 
human error, weather and other natural causes, and physical and cyber attacks. The 
directive contends that addressing these vulnerabilities require flexible and evolutionary 
approaches for the public and private sectors. Frequent assessments are made of critical 
infrastructures’ reliability, vulnerability and the threat environment because the threats to 
infrastructures will continue to change and protective measures and responses must be 
robust and adaptive. 

NSA is charged with the National Manager responsibilities and assesses 
U.S. Government systems for interception and exploitation, disseminates threat and 
vulnerability notices, establishes standards, and conducts research and development in 
areas of security product evaluations. 

g. The E-Government Act of2002 (Public Law 107-347) 

The E-Govemment Act of 2002 recognizes the importance of information 
security to the economic and national security interests of the United States. Title III of 
the E-Government Act, entitled the Eederal Information Security Management Act of 
2002 (EISMA), tasks NIST with the responsibility for standards and guidelines. This 
includes development of the standards to be used by all federal agencies to categorize all 
information and information systems collected or maintained by each agency. This is 
based on the objectives of providing appropriate levels of information assurance 
according to a range of risk levels and guidelines to recommend the types of information 
and information systems that should be included in each category 

h. Federal Information Security Management Act of2002 (FISMA) 

EISMA directs federal agency heads and their Chief Information Officers 
(CIOs) to ensure that there is an information security program in place as well as trained 
personnel to administer the program. A great emphasis is placed on fully integrating 
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security into the existing and future business processes. Each management official, 
typically referred to as the Designated Approval Authority (DAA), is required to 
authorize each system for operation with a formal certification and accreditation (C&A) 
process. The certification and accreditation process is required on all federal information 
systems. This process is intended to ensure that the appropriate security controls are 
implemented and are operating as intended. FISMA further requires that agency systems 
be certified and accredited to continue IT operations, which includes those federal 
systems subject to HIPAA compliance. 

Agency heads are responsible for providing information security 
protections regarding the magnitude and risk of harm resulting from unauthorized access, 
use, disclosure, disruption, modification, or destruction of data or information systems. 
Requirements include periodic testing and evaluation of the effectiveness of information 
security policies, procedures, and practices, performed with a frequency depending on 
risk, but no less than annually, to ensure that they are effectively implemented. These 
procedures are required for detecting, reporting, and responding to security incidents, and 
are consistent with standards and guidelines, including the mitigation of risks associated 
with such incidents before substantial damage occurs. 

2. DoD Requirements 

a. DoD 5200.28-STD - Department Of Defense Trusted Computer 
System Evaluation Criteria. 

The DoD 5200.28-STD, paragraph 2.2.3.2.1 directs department heads to 
test security protection mechanisms to confirm they work as claimed in the system 
documentation and to search for obvious flaws that would allow the bypass of security 
mechanisms, permit a violation of resource isolation, and allow unauthorized access to 
the audit or authentication data 

b. DoD Instruction 8500.2 - Information Assurance (lA) 
Implementation 

DoD Instruction 8500.2, paragraph E3.3.10. requires each DoD 
component’s information assurance (lA) program to regularly and systematically assess 
their lA posture with regard to DoD component-level information systems, and the DoD 
component-wide lA services and supporting infrastructures via combinations of self- 
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assessments, independent assessments and audits, formal testing and certifieation 
activities, host and network vulnerability testing, as well as lA program reviews. 

3, Command Requirements 

a. Military Health System (MHS) Information Assurance (lA) 
Policy/Guidance Manual 

The provisions of this policy apply to all MHS components, military 
personnel, DoD civilians, and contractors, who manage, design, develop, operate, or 
access DoD information systems, and the TRICARE Management Activity (TMA)- 
developed and operated information systems, or access DoD data. MHS Components 
include: Service Medical Departments, TMA Directors, TMA Centrally Managed 
Systems, and TRICARE contractors. 

Risk assessments are to be conducted whenever significant and major 
changes occur or when new threats are identified to the DoD IS or the IS operating 
environment. MHS Components are directed to attempt to exploit network security 
vulnerabilities using penetration testing during the C&A process, or more frequently as 
required by the MHS lA Program Office. Penetration tests on DOD information systems 
will be conducted by the MHS lA Program Office, in coordination with the appropriate 
Service, to verily the adequacy of security countermeasures in place. 

Vulnerability Assessments performed on MHS Components will identify 
system and network vulnerabilities through use of vulnerability assessment tools. 
Vulnerability assessments are to be conducted on the network and critical servers and 
systems at least annually. Additionally, the Systems Administrator (SA) and the 
Information Systems Security Officer (ISSO) obtain and run vulnerability assessment 
software on automated information systems and networks monthly. 

The MHS Components will incorporate a comprehensive process to audit, 
detect, isolate, and react to intrusions, service disruptions, and incidents that threaten the 
security of operations. Individual sites are required to review audit records for DOD 
information systems on a monthly basis or more frequently when deemed necessary. 
Continuous security monitoring will be performed within each MHS Component. The 
information system owners will ensure the information systems they are responsible for 
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are regularly monitored, that system reeords are reviewed on a weekly basis, and that all 
DOD information systems are proteeted by Intrusion Deteetion Systems (IDS). 

b. Bureau of Medicine and Surgery (BUMED) Information 
Assurance Information Systems Security Policy Manual 

A primary funetion of the Bureau of Medieine and Surgery (BUMED) is 
to ensure the National Command Authority has a healthy fighting foree with a supporting 
eombat-ready health eare system. The inherent sensitivity of the BUMED healtheare 
information systems is aseertained by the eoneerns for individual privacy and the 
integrity of the personal and medical information processed, as well as the availability of 
the information systems that support the Navy’s health care programs. 

The BUMED Information Systems Security Program was implemented to 
ensure required protective measures are implemented to protect BUMED information 
systems against unauthorized modification, disclosure, destruction, and denial of service 
throughout all system life cycle phases. The document establishes the security policy for 
protecting the data, services, and resources related to development, maintenance and 
operations involving the systems and networks in Claimancy 18 activities, which are 
comprised of approximately 400 commands. Each system's level of security must protect 
the confidentiality, integrity, and availability of the information. Specifically, the 
document requires that each system undergoes periodic monitoring to test for known 
operating system vulnerabilities. It further recommends that every open port should be 
associated with a known application and that all other should be terminated and that 
regular monitoring of system logs for suspicious activity should be conduced. Einally, the 
policy recommends the use of available tools to periodically audit systems, especially 
servers, to ensure that there have been no unauthorized or unknown changes to the file 
system, registry, or user account database. 

4. Comparative Summary of Information Assurance Policies 

Table 4 below shows a comparison of the information assurance policies 
described above. The Vulnerability Scanning Requirements column indicates how often 
vulnerability assessments are required, at a minimum. Per the background information 
reviewed in this chapter, none of the minimum requirements are sufficient in today’s 

networked computing environment, as the window between new threats decreases. 
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Policy 

Vulnerability 

Valid in Current 


Scanning 

Operating 


Requirement 

Environment 

Federal 



Computer Act of 1987 

None 

No 

Privacy Act of 1974 

None 

No 

HIPAA 

Periodic 

No 

FISMA 

Annually 

No 

0MB Circular A-130 

None 

No 

PDD 63 

Frequently 

No 

DoD 



DoD 8500.2 

Regularly 

No 

Organizational 



MHS 

Monthly 

No 

BUMED 

Periodically 

No 


Table 2. Comparative Summary of Information Assurance Policies 


After a thorough review of the aforementioned policies and directives, it can be 
concluded that the MHS policy is the most stringent attempt made to require Naval 
Medicine activities to properly address or meet the current information assurance threats. 
The overarching policies are vague at best and should be revised as soon as feasible. 
Retired Vice Admiral Arthur Cebrowski, Director of Force Transformation for the Office 
of the Secretary of Defense stated that “...trends, which futurists call ‘perfectly 
predictable surprises’ - when the rate of transactions exceeds the resources, then policy 
will change - are already showing, and aiming toward networks and networking 
behavior”( Roosevelt, 2004). As task loads continue to increase, IT managers will have 
to voice their concerns about the technical issues confronting them. Alerting policy 
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makers, managers and strategists are parts of the solution (Azari, 2003). Without the 
awareness of an ever-inereasing responsibility, management may never know what is 
required. MHS and will and ean assist in evaluating network security assessments, but 
coordinating and scheduling full scale network security audits may take some time as the 
technical experts on staff to perform such task is limited in number. 

I. DEFENDING YOUR INFORMATION ASSETS 
1, Awareness Training 

a. Continuing Education 

Continuing education is a must in information technology. New systems 
hardware, software, user features and enhanced capabilities continue to alter our 
connected world, and technology continues to push the boundaries of physics and space. 
Continuing education does not necessarily mean that increases in training budgets are 
required. There are a number of free vendor seminars and publications that yield 
significant amounts of information security-related training. John Saunders has provided 
an excellent Web page at http://www.iohnsaunders.com/securitv.htm that maintains a 
plethora of information security topics. Table 3 displays the key categories found on that 


page. 

C’oiiiputer Security Links (asofM»y:oo4) 


Antivirus 

Attacks & 
Vulnerabilities 

Assessment 

Biometrics 

Cryptoaraphy 

Education 

Firewalls 

Forensics 

"Free" Items 

Incident Reportina 

Intrusion 

Detection 

Laws 

Miscellaneous 

Network 

Penetration 

Testina 

Online 

Publications 

Ooeratina System. 

Web Server & 
Router Security 

Oraanizations & 

Certification 

Oversiaht- 

Federal 

Public Kev 
Infrastructure 

Research & 

University 

Risk 

Manaaement 

Virtual Private 

Networks 

Wireless Access 

and Security 

Other General 

Security Links 


Table 3. Computer Security Links 

Federal Computer Weekly often has current IT news and trends that are 
relative to the U. S. Federal Government. Although the title is misleading, this resource is 
a commercial entity. URL: http://www.fcw.com/links/legislation/techleg.asp 

The U.S. Government also hosts many helpful links. The following are 
among the more prevalent resource materials available online. 
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The National Institute of Standards and Teehnology (URL: 
http://www.nist.gov/) is a repository of laws, statutes, aets. Executive Orders, and 
multiple policies concerning information technology issues relevant to the federal 
government. 

The Office of Homeland Security, The White House (URL: 
http://www.whitehouse.gov/homeland) is the newest department in the Executive Branch 
that possesses the teeth to affect information technology components and processes. 

The Sixty-Minute Network Security Guide (Eirst Steps Toward a Secure 
Network Environment) was published by the Systems and Network Attack Center 
(SNAC) of the National Security Agency. An E-mail request should be sent to 
SNAC.Guides@nsa.gov for the current URL of this valuable tool. 

The Defense Information Systems Agency is another valuable site that 
should not be overlooked. The Information Support Environment link 
http://iase.disa.mil/eta/ provides free video and training/tutorial CDs on some of the 
hottest topics in information security today. The information assurance videos are great 
for the required annual information security refresher training. 

A number of other informative sites are available for researching 
vulnerabilities and threats that have been identified for specific systems and services. 
They can be reviewed at the following sites: 

• Security Eocus www.securityfocus.com 

• Incidents.org www.incidents.org 

• InfoSysSec www.infosvssec.com 

• Packet Storm www.packetstormsecuritv.org 

b. Annual Training 

Annual training is a reminder to all organizational personnel that security 
is an individual responsibility. Not all personnel are security engineers, but the basics 
should always be included in such training. This include, but are not limited to social 
engineering methods, which remains one of the most effective methods that attackers 
utilize to gain access to an organization’s information assets; password management. 
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locking system desktops, reporting suspicious activity or system files including 
unsolicited email attachments. 

c. Professional Training 

Professional training is not for everyone, but a number of organizations 
provide information security training programs and certifications. Some are vendor- 
specific and others operate as a non-profit organization. Most professional certifications 
require a test and/or practical demonstration of knowledge in a wide range of domain- 
specific areas of information security. Some of the most recognized security 
certifications are: 

Certified Information Systems Security Professional (CISSP) - Visit 
http://www.isc2.org for additional information. 

System Administration, Networking, and Security (SANS) - Visit 
http://www.sans.org/ for additional information. 

Vendor-specific: 

Cisco certifications are organized in 3 major categories; Associate, 
Professional, and Expert levels of expertise. Visit http://cisco.netacad.net/public/ for 

additional information. 

Microsoft Certifications are arranged for systems administrators, 
application developers, solutions developers, systems engineers, and database 
administrators. Additional details can be found at Microsoft’s Web site, located at 
http://www.microsoft.com/education/msitacademv/WorldWide/Default.aspx . 

d. Experience-based 

Education can only be supplemented by time and experience. Although 
experienced security personnel have existing certifications, continued training is required 
to expand their knowledge base current. More experienced personnel should be 
scheduled for advanced training whenever feasible. 


28 






2, Vulnerability Assessments 

a. Purpose 

Vulnerability assessments are an effective way to identify potential 
vulnerabilities in a system or network. These exercises of security evaluation usually 
employ common attack methods that an adversary may use in an attempt to access 
information systems of interest. These methods may range from a simple IP scan to 
identifying resources that utilize services with known vulnerabilities or unpatched 
systems for future exploits. The end goal of vulnerability assessments is to report system 
weaknesses to the owner for resolution or to the attacker for a future exploit. 

Vulnerability assessments are performed for a number of reasons, but they 
are not considered a simple task and usually require special knowledge to perform. Fears 
of corrupting or breaking existing systems are generally the reason they are not a standard 
inter-organizational practice. They are normally composed of multiple rather than one 
aggressive attack methods and when they are performed, it is done when activity periods 
are slow as the networked systems or the network as a whole can be disabled during the 
process. Vulnerability assessments are often performed during certification and 
accreditation assessments or whenever a test of organizational intrusion detection and 
response capabilities are desired. 

b. External 

External vulnerability assessments originate from the platform on which a 
true adversary would likely attack from. This type of attack tests the abilities of the 
firewall and router filtering capabilities, all the systems that are accessible from the 
outside, such as web and mail servers, as well as gateway-specific controls that may 
assist to block such attempts. These types of tests are seldom performed due to the 
complexity and legal situations that may arise from such vulnerability assessments, 
especially in DoD networks, where other entities outside of the organization may be 
monitoring network activities. External attack scenarios take a considerable amount of 
coordination by all parties concerned. 
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c. 


Internal 


Internal vulnerability assessments are better managed within specified 
boundaries of the network. These attacks can be directed at a network segment more 
easily and consume less resources in the process. Tests of network segments are less 
costly, more controlled, and safer to perform in regards to network stability. 

d. Network Surveys 

Network surveys promote a more comprehensive method of testing the 
overall security posture of a network. These foot printing and scanning exercises provide 
an insight to determining which resources are available for testing purposes. Mapping or 
surveying most generally yields domain names, server names, Internet service provider 
information, Internet protocol (IP) addresses of individual hosts, and their interconnecting 
devices. The Nmap tool is very effective for this type of discovery. Nmap can 
differentiate which operating systems are running on a network as well as the types of 
packet filters or firewalls are in use. Additional details can be found by visiting the Nmap 
hyperlink posted in Table 7. 

e. Limitations of Vulnerability Assessments 

If vulnerability assessments are going to be initiated to simulate a real 
attack, they should be conducted as "black box" exercises. In a real attack, the attacking 
agent would not normally possess intimate information about the system being tested. 
Knowing about the system specifics would actually invalidate the test before it could 
begin. It is easy to imagine that an attacker already knowing administrator passwords 
and how your network was configured would not really be testing anything except for 
their personal skills. A simulated attack will only identify the problems that it is designed 
to look for. If the tools are not configured to seek a system feature or service, it will not 
produce any information about its level of security or insecurity. It is also important to 
remember that vulnerability assessments would seldom, if ever, provide information 
about vulnerabilities that have not yet been discovered and well documented within the 
security community (one must know the “signature” of what they are looking for... few, 
if any, heuristic tools are available in this area). Furthermore, if there are no instances 
where vulnerabilities are identified after the vulnerability assessment is complete, it does 
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not necessarily mean that a network is secure. Assessments are a “snap-shot” in time, 
and can become obsolete within days as new vulnerabilities are discovered by 
information assurance professionals. If vulnerabilities are discovered, as they often are, it 
is imperative that the corrective configuration settings and or patches are applied as soon 
as possible. The effort required to remediate vulnerabilities can be quite substantial. 
More often than not, a vulnerability report collects dust before corrective actions are 
implemented, which often means that more vulnerabilities have likely been reported and 
the system will not be any more secure after the earlier noted changes are applied. It 
must be remembered that one vulnerable platform is all that an attacker needs to 
influence a network operating environment. 

f. Unexpected Consequences Derived From Testing 

Vulnerability assessments can have serious consequences for the network 
on which they are run. If badly conducted it can cause congestion and systems crashing. 
It is, therefore, vital to have consent from the management of an organization before 
conducting vulnerability assessments on its systems or network. If the issue of timing is 
not resolved properly, it could be catastrophic to an organization. Imagine conducting a 
denial-of-service ‘test’ on a university on the day its students take their online 
examinations. This is an example of poor timing as well as lack of communication 
between the vulnerability assessors and the university. Good planning and preparation 
will help avoid such bad practices. 

3, Automated Tools 

a. DoD Approved 

Government off-the-shelf (GOTS) vulnerability scanning software is 
available from the Defense Information Systems Agency (DISA) at no cost to all 
government agencies. There are two versions of Security Profile Inspector (SPI), for 
Windows NT (SPI-NT) operating systems and for Unix Networks (SPI-NET). Both 
versions can be retrieved from: http://www.cert.mil/resources/securitv tools.htm 

The Department of Homeland Security (DHS) offers another no-cost 
service to federal agencies. Patch Authentication and Dissemination Capability (PADC) 
is a service that allows agencies to retrieve information on trusted and authenticated 
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patches for their speeifie operating systems. Subseriptions must be requested from the 
DHS’s Federal Computer Ineident Response Center (FedCIRC). Of the 2000 aeeounts 
available, only 47 ageneies had aetive subseriptions as of 10 September 2003. Other 
pateh management solutions may offer expanded eapabilities, but they are not free of 
eharge. 

b. Non-DoD Approved 

A reeent study published on 26 June 2003 by Kevin Novak provides a 
great level of detail in regards to vulnerability assessment seanners. The study examined 
11 of the most prevalent vulnerability assessment seanners on the market. The features, 
eapabilities, eompany information, and eosts assoeiated with those systems are listed in 
Tables 4 through 6. 
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VENDORS AT A GLANCE 



Table 4. Vulnerability-Assessment Tools: Vendors at a Glance 

From http://www.nwc.com/1412/1412f2.html 
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From http://www.nwc.com/1412/1412f22.html 
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Table 6. Vulnerability-Assessment Tools: Report Card. 
http://www.nwc.eom/1412/1412f213.html 
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c. Vulnerability Assessment Tools 

There are a great number of other tools available that are designed to 
automatically discover vulnerabilities. Nessus is a scaning utility that remains a favorite 
among attackers and can be found at http://www.nessus.org. Nessus possesses the 
capability of remotely auditing a network and reporting the existing vulnerabilities. A 
short abbreviated list of other commonly used tools with a brief description of their 
capabilities is provided below. 

1) Information Gathering Tools: 

• Nmap - Network and port scanner with operating 
system discover 

• Hping - Port scanning tool 

• Netcat - Obtains service banners and versions 

• Firewalk - Useful for determining a firewall access 
control list (ACL) 

• Ethereal - Useful for monitoring and logging traffic 
returning from maps and scans 

• Icmpquery - Used to determine target systems time 
and network mask used to hide real addresses 

• Strobe - A useful port scanning utility 

• Superscan - A Windows port scanning tool 

• RPCDump- Command line tool that performs 
queries on Remote Procedure Call (RPC) endpoints 

• Netstat - Shows active TCP connections, open 
ports, Ethernet statistics, and the IP routing table 

The Eoundstone Company is another free defense resource site that also 
offers a comprehensive list of tools for security risk management and vulnerability 
assessments. The tools offered freely to the public are the ones used in the field by its 
consultants. Visit them at http://www.foundstone.com/ and click the resources link to 
view the available tools. 

Another type or method of vulnerability assessment involves password 
breaking, also referred to as password cracking. Again, these are automated tools that are 
simple to use and are limited only by processing power. Even on standard use personal 
computers, a password cracking utility can process more than 100,000 guesses per 
second. One of these utilities is especially effective against passwords required for 
remote access systems allowing Telnet and ETP transfers, since it does not require the 
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password file off a computer, as do the first three mentioned below. The following lists a 
few of the password cracking methods and tools used today. 

• Dictionary Attack - Uses a word list or dictionary file and can be modified to 
incorporate multiple languages. One standard dictionary attack takes a few 
minutes to test every word. It is a fast method that is often very effective where 
password policies are not enforced. 

• Hybrid Crack - Tests for passwords that are variations of the words in a 
dictionary file. It consumes more time, but yields more results. 

• Brute Force - This method uses a variety of tests for passwords that are made 
up of characters and numbers by performing every combination possible. This 
method is most effective and will break anything given enough time, time being 
the key ingredient. A password of eight characters or more could take from days 
to millions of years to crack. 

• Brutus- This tool is used to automatically crack telnet and ftp accounts. Fast, 
effective method to demonstrate to management why those types of remote access 
are not a novel idea any longer. Brutus is not included in Table 4, but is available 
at http://www.hoobie.net/brutus 

Please refer to Table 7 for additional tools that grant access and escalation 
of privilege. LOphtcrack and John the Ripper are two of the most appropriate tools for 
password cracking. 

d. Tools and Information Available to the General Public 

Table 7 is a brief compilation of tools available to network defenders and 
attackers to determine where an organization’s strength and weaknesses are, created by 
the authors of the "''Hacking Exposed” series of books. 


General Security Tool Sites 


Hackersclub 

http://WWW. hackersclub.com 

NewOrder j 

http: / / neworder. box.sk 

Security-Focus 

http://www.securitvfocus.com 

Technotronic 

http://WWW. technotronic.com 


Countermeasure Tools 


BlackICE by 
NetworkICE 

http://WWW. networkice.com 

CyberCop Monitor 
by Network 

http: //WWW. nai .com 
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Associates Inc. 


Hidden Object 
Locator 

htto://WWW. netwarefiles.com/utils/hobiloc.zip 


IppI 

http: //WWW. via.ecD.fr/~huao/iDDl/ 


ITA from Axent 

http://WWW. axent.com 

Kane Security 
Monitor 

http://WWW. intrusion.com 

Netguard 

http://WWW. Genocide2600.com/~tattooman/unix-loaaers/netauard- 

I.O.O.tar.az 

Network Flight 
Recorder 

http: / / WWW. nf r. net 

Perro (formerly 
Proto log) 

http:/ /WWW. ariana.com/dieao/linux/Drotoloa/index.html 


Psionic Portsentry 
from the Abacus 
project 

http: //WWW. Dsionic.com/abacus/ 


RealSecure by 
Internet Security 
Systems (ISS) 

http: //WWW. iss. net 

Scanlogd 

http: //WWW. ODenwall.com/scanload/ 


Secured by 

Memco 

http://WWW. memco.com 

Secure Shell (SSH) 

httpi/Zwww.ssh.fi 
http:/ /WWW. f-secure.com 

SessionWall-3 by 
Abi rnet/P lati n u m 
Technology 

http://WWW. abirnet.com 


Denial of Service 


Land and 
Latierra 

http: / /WWW. rootshell.eom/archive-i457nxiqi3aa59dv/199711/land, c.html 

http: //WWW. rootshell.com/archive-i457nxiqi3aa59dv/199711 /latierra.c.html 


Portfuck 

httD://www.staraazer.net/~flatline/filez/Dortfuck.ziD 


Smurf & Fraggle 

http: //WWW. rootsheU.com/archive-i457nxiq13aq59dv/199710/smurf.c.html 

http: //WWW. rootsheU.com/archive-i457nxiqi3aq59dv/199803/fraaale.c.html 


Synk4 

http: //WWW. iabukie.com/Unix Sourcez/svnk4.c 
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Teardrop, 
newtear, bonk, 
syndrop 

http: / / WWW. rootshell. com/archive- 

i457nxiqi3sq59dv/199711 /teardrop.c.html 

http://www.rootshell.com/archive-i457nxiqi3sq59dv/199801 /newtear.c.html 

http://www.rootsheU.com/archive-i457nxiq13sq59dv/199801 /bonk.c.html 

http://www.rootsheU.eom/archive-i457nxiqi3sq59dv/199804/svndrop.c.html 



Enumeration Tools 


Bindery 

http:/ /WWW. nmrc.ors/files/netware/binderv.zip 


Bindin 

ftp://ftp.edv-himmelbauer.co.at/NoveU.3x/TESTPR0G/BINDIN.EXE 


Epdump 

http://WWW. ntshoo.net/securitv/tools/def.htm 


Finger 

fto://fto.cdrom.com/.1/noveU/finser.zio 


Legion 

ftp://ftp. technotronic.com/rhino9-oroducts/lesion.zip 


NDSsnoop 

fto://fto.iae.univ-ooitiers.fr/oc/netware/UTIL/ndssnooo.exe 


NetBIOS Auditing Tool 
(NAT) 

ftp://ftp. technotronic.com/microsoft/nat10bin.zip 


Netcat by Hobbit 

http: //WWW. I0oht.com/~weld/netcat/ 


Netviewx 

http: //WWW. ibt.ku.dk/iesoer/NTtools/ 


Nslist 

htto://wvw.nmrc.ors/files/snetware/nut18.zio 


On-Site Admin 

fto://fto.cdrom.com/.1 /novell/onsite.zip 


Sniist 

ftp://ftp.it.ru/oub/netware/util/NetWare4.Toos/snlist.exe 


Somarsoft (dumpaci, 
dumpreg, etc.) 

htto://38.15.19.115/ 

userZsid and sidZuser 

http://www.chem. msu.su: 8080/~rudnvi/NT/sid.txt 


Userdump 

fto://fto.cdrom.com/.1 /noveU/userdump.zip 


Userinfo 

fto://fto.cdrom.com/.1 /noveU/usehnfo.zip 



Footprinting Tools 

ARIN database 

htto://www.arin.net/whois/ 

Cyberarmy 

http: //www.cvberarmv.com 

Dogpile (meta search engine) 

http: //WWW. dosoUe.com 

DomTools (axfr) 

http://www.domtools.com/pub/domtools1.4.0.tar.sz 
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Ferretsoft 

htto: //WWW. ferretsoft.com 

Sam Spade 

httD://www.samsDade.ora 

Securities and Exchange Commission 
(SEC) 

htto: //www.sec.aov/ 


USENET Searching 

http://www.deia.com 

http://www.doaoile.com 

VisualRoute 

htto: //WWW. visualroute.com 

WHOIS database 

htto: / / WWW. networksolutions. com 

WS_ Ping Pack Pro 

htto: //WWW. ioswitch.com 


Gaining Access 


LOphtcrack's 

Readsmb 

htto: //WWW. I0oht.com/ 

Legion 

htto: //WWW. rhino9.com 

NetBios Auditing 
Tool (NAT) 

ftp://ftp. technotronic.com/microsoft/nat10bin.zip 


Nwpcrack 

htto: www.nmrc.ora/files/netware/nwocrack.zio 


SMBgrind by NAI 

Included with CyberCop Scanner from Network Associates 
(http://www.nai.com) 

Sniffit 

htto://newdata.box.sk/neworder/a/sniffit.0.3.2.tar.az 


SNMPsniff 

htto://www.AntiCode.com/archives/network-sniffers/snmosniff-1 O.taz 


THC login/telnet 

htto://thc.pimmel.com/files/thc/thc-lh11 .zip 



Privilege Escalation and Back Door Tools 


Elitewrap 

htto: //WWW. multimania.com/troianbuster/elite.zip 



Getadmin 

htto: //WWW. ntsecuritv.net/securitv/aetadmin.htm 



Hunt 

htto://www.cri.cz/kra/index.html#HUNT 

Imp 

htto: //WWW. wastelands.aen.nz/ 

Invisible Keystroke Logger 

htto: //WWW. amecisco.com/iksnt.htm 

Jcmd 

h tto: / / wvw. i rbsof t wa re. co m 
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John the Ripper 

httD://www.openwaU.com/iohn/ 


Netbus 

htto: / /WWW. netbus. ora 

Netcat 

httD://www.lODht.com/netcat 

NTFSDOS 

htto://www.svsinternals.com 

NT user 

htto: / / WWW. Dedestalsoftware.com 


Pandora by NMRC 

htto:/ /WWW. nmrc.ora/pandora/download.html 


PwdumpZ 

htto: //WWW. WebsDan.net/~tas/DwdumD2/ 


Revelation by Snadboy 

http://www.snadbov.com 

Sechole 

http://WWW. ntsecuritv.net/securitv/sechole.htm 


SNMPsniff 

htto://packetstorm.harvard.edu/sniffers/snmpsniff-l .O.tar.az 


Unhide 

http://www.Webdon.com 

Virtual Network Computing 
(VNC) 

http:/ /WWW. uk. research, att.com/vnc 



Pilfering 

File Wrangler 

http: //WWW. tucows.com 

PowerDesk's ExplorerPlus 

http: //WWW. miienix.com/Dowerdesk98. asp 


Revelation 

http: //www.snadbov.com 


Rootkits and Cover Tracks 


Cygwin Win32 (cp and touch 
commands) 

http://www.cvanus.com 

Wipe 

ftD://ftD.technotronic.com/unix/loa-tools/wiDe- 

I.OO.taz 

Zap 

ftD://ftD.technotronic.com/unix/loa-tools/zaD.c 



Scanning Tools 


Bind View j 

http://WWW. bindview.com 

Chknull 

http:/ /WWW. nmrc.ora/files/netware/chknull. zip 


CyberCop Scanner by NAI 

http://WWW. nai.com 
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Firewalk 

http: / / WWW. packetf actorv. net/firewalk/ 


Fping 

http://oacketstorm. harvard.edu/ 


HackerShield by Bindview 

http://www.bindview.com/netect 


Hping 

http://WWW. kvuzz.ora/antirez/ 

InspectorScan by Shavlik 

http://v/ww.shavlik.com 

Internet Scanner by ISS 

httpi/www.iss.net 

Kane Security Analyst 

http: //WWW. intrusion.com 

Network Mapper (Nmap) 

http://v/ww. insecure, ora/nmap 

NTInfoscan 

http://www.infowar.co.uk/mnemonix/ 


Finger 

ftp://ftp. technotronic.com/rhino9-oroducts/oinaer.zio 


Scan 

http://WWW. orosolve.com 

Solarwinds 

http: / /WWW.solarwinds. net 

Strobe 

http: //WWW. hack-net.com/caibin/download.cai?strobe-1 03.taz 


Udpscan 

ftp://ftp.technotronic.eom/unix/network-scanners/udpscan.c 


WebTrends Security 

Analyzer by WebTrends 

http: //WWW. Webtrends.com 

WS_Ping Pack Pro 

http: //WWW. ioswitch.com 


War Dialing Tools 


PhoneSweep by Sandstorm 

http: / /www.sandstorm. net 

THC ! 

http://v/ww. infowar.co.uk/thc/ 


ToneLoc 

http://www.hackersclub.com/km/files/pfiles/Tl110.zip 



Table 7. Network Defense and Attaek Tools and Links 


From http://www.haekingexposed.eom/tools/tools.html . 

4, Proactive Measures 

a. Systems Configuration 

Vulnerability scanners will alert system owners of potential weaknesses 

within their information system, but maintaining the appropriate systems configuration 
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alleviates many of the vulnerabilities found in unmanaged systems. Applieations and 
serviees required to operate the system should be evaluated to determine whieh ports and 
protoeols are required for functionality. All unused ports and services should be 
terminated. Many of the findings that are derived from vulnerability assessments address 
unnecessary open port and service issues. The majority of applications and operating 
systems on the market today are loaded with default settings focused on providing the 
customer with all available services included in the software. Many of those services 
installed by default are never required and place the system at a higher level of risk as 
soon as it is connected to the Internet. 

NIST has produced a number of Special Publications to assist in 
information assurance tasks. The following table is referenced in the draft version of 
NIST SP 800-66, An Introductory Resource Guide for Implementing the Health 
Insurance Portability and Accountability Act (HIPAA) Security Rule. This is a great 
resource in its entirety. Appendix E has an extensive HIPAA Security Rule/FISMA 
requirements crosswalk table that breaks down every element required for compliance 
with the federal mandates. See Table 8 for a quick review of what NIST has to offer. 


NIST Publication 

Title 

FIPS 140-2 

Security Requirements for Cryptographic 

Moduies 

FIPS 199 

Standards for Security Categorization of Federai 
Information and Information Systems 

NISTSP 800-12 

An Introduction to Computer Security: The NIST 
Handbook 

NISTSP 800-14 

Generally Accepted Principles and Practices for 
Securing Information Technoiogy Systems 

NISTSP 800-16 

Information Technoiogy Security Training 
Requirements: A Rote- And Performance-Based 
Modei 

NISTSP 800-18 

Guide for Deveioping Security Pians for 
Information Technoiogy Systems 

NIST SP 800-26 

Security Seif-Assessment Guide for Information 
Technoiogy Systems 

NIST SP 800-27 

Engineering Principies for Information 

Technoiogy Security (A Baseiine for Achieving 
Security) 

NIST SP 800-30 

Risk Management Guide for Information 
Technoiogy Systems 
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NIST SP 800-34 

Contingency Planning Guide for Information 
Technology Systems. 

NIST SP 800-35 

Guide to Information Technology Security 
Services 

NIST SP 800-36 

Guide to Selecting Information Security 

Products 

NIST SP 800-37 

Guide for the Security Certification and 
Accreditation of Federal Information Systems 

NIST SP 800-42 

Guideline on Network Security Testing 

NIST SP 800-44 

Guidelines on Securing Public Web Servers 

NIST SP 800-47 

Security Guide for Interconnecting Information 
Technology Systems 

NIST SP 800-50 

Building Information Technology Security 
Awareness and Training Program 

NIST SP 800-53 

Recommended Security Controls for Federal 
Information Systems 

NIST SP 800-55 

Security Metrics Guide for Information 

Technology Systems 

NIST SP 800-56 

Recommendation on Key Establishment 

Schemes 

NIST SP 800-57 

Recommendation on Key Management 

NIST SP 800-59 

Guideline for Identifying an Information System 
as a National Security System 

NIST SP 800-60 

Guide for Mapping Types of Information and 
Information Systems to Security Categories 

NIST SP 800-61 

Computer Security Incident Handling Guide 

NIST SP 800-63 

Recommendation for Electronic Authentication 

NIST SP 800-64 

Security Considerations in the Information 

System Development Life Cycle 


Table 8 . NIST Publications Referenced in NIST SP 8 OO -664 


Figure 4 illustrates how the NIST publications relate to the essential 
elements for creating and managing an information assurance security program. 
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Planning 

800-12, 800-14. 
FIPS 199. 800-60, 
800-18, 800-59 


Implementation 

800-16, 800-50, 800-30, 
800-34. 800-35, 800-36. 
800-37, 800-53, 800-61, 
800-47. 800-55. 800-64 


I 


Technical and IT Infrastructure 
Specific Guidance 

FIPS 140-2, 800-27, 800-42, 
800-44, 800-56, 800-57, 
800-61, 800-63 


Assessment 

800-26, 800-53, 
Internal/External 
audits and reviews 



Figure 4. NIST Publications 

J. POTENTIAL BENEFITS TO NAVAL MEDICINE 

1, Navy Marine Corps Intranet (NMCI) Implications 

a. Maintenance for Non-Qualifying Systems 

Although the NMCI initiative will assume operations for the majority of 
systems with the Navy and Marine Corps, and therefore the security associated with 
them, some exceptions to the assumption of individual networks will occur. Many of the 
legacy programs will not meet the required certification and accreditation status needed 
to operate on the NMCI network. Until they are replaced or incorporated into other 
qualifying systems, the need to manage those systems vulnerabilities will remain a 
requirement. 

2, Greater Assurance of Due Diligence in Personal Privacy Issues 
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Federal legislation has recently mandated that executive staff members, especially 
the Commanding officers of those organizations, are responsible for the safekeeping of 
personal data stored on information systems. Ignoring that responsibility may require 
that their negligence be penalized with significant fines and/or imprisonment. An 
effective patch management program will more likely demonstrate due diligence should a 
compromise occur than would having none at all. 

3, Estimated Savings in Personnel Costs 

Gartner group estimates that a 1000-unit server farm costs $300,000 per year to 
perform patch management tasks. The same server population would cost $50,000 to 
implement an automated solution (Schroder et al, 2003). This question will be covered in 
greater detail in Chapter V, Conclusions and Recommendations. 

4, Significant Reductions in Vulnerabilities 

The overall benefit derived from performing continuous vulnerability assessments 
across the entire network will not only alert administrators of existing weaknesses, but 
will save them numerous hours in reconfiguration efforts required to recover from 
compromised systems. When considering the legal responsibilities facing organizations 
in today’s network-centric environment, can anyone really afford to leave their systems 
unprotected? According to Security Alert Consensus www.sans.org/newsletters/sac , 
there are approximately 1000 new operating systems and applications vulnerabilities 
reported each year, which is roughly 83 new vulnerabilities per month (Shipley, 2003). 
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III. RESEARCH METHODS 


A, WORK PLAN 

This project utilizes an applied research methodology, including both primary and 
secondary research. This research is limited to Naval Medicine personnel who are 
directly responsible for information systems operations. 

B, SECONDARY 

Secondary research was obtained from online sources, including the World Wide 
Web and the Dudley Knox Library archives at Naval Postgraduate School. These efforts 
confirm the current technologies utilized by other information-centric organizations and 
seek the most effective employment techniques (scheduling, automation, etc.). 
Interviews with industry and government information assurance professionals aid in 
determining a return on investment, should the recommended policies and practices be 
accepted. 

C, PRIMARY 

The primary research begins with an evaluation of the current Naval Medicine 
network vulnerability management policy and practices. A comparison to Federal 
Information Security Management Act (FISMA) and other federal/service policies is 
reviewed to confirm or recommend current policy modification. In addition to policy 
review, the information assurance (I A) methodologies suggested by NS A will be 
recommended, if applicable, to enhance existing lA practices. A Web-based survey 
seeking general information in regards to lA policy, known systems compromises, 
current vulnerability scanning methods, and patching practices will be posted on the 
Naval Medicine Intranet portal referred to as Naval Medicine Online (NMO). The results 
provide insight into the effects of current policy, tools and techniques used by 
information technology managers utilized to protect information system assets. 
Submitted survey responses are anonymous and solely intended to survey current policy 
practices, overarching policy adherence and current vulnerability assessment practices. 
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D, SURVEY INSTRUMENT DEVELOPMENT 

The survey instrument is designed for comprehensibility to all participants while 
capturing the information necessary to validate or invalidate the premise: Seventy-five 
percent of Naval Medicine’s known information systems compromises were not 
protected by the available vulnerability patch(es). The survey is compiled with an All- 
Points-Anchored response option for each of 30 questions. Response options for 
Questions 5 through 30 will be coded as 1 through 4 for statistical analysis. The survey 
development tool within the Naval Medicine Online portal will be utilized to create an 
HTML-based survey. A survey key, generated and delivered to each Naval Medicine 
CIO, ensured that only one survey was submitted by each organization. An emphasis in 
creating a short, easy-to-understand instrument was utilized to encourage participation 
and to facilitate ease in completion. Each of the 30 questions has supplemental 
descriptors for clarification. Additionally, each question was placed in a logical 
progression, while a best effort approach was made to keep response categories as close 
as possible to similar response categories. Respondents were instructed to select one 
response for each applicable question. 

The survey instrument displayed four response sections in a linear table format. 
The first portion of the survey. Questions 1 through 4, was used for demographic analysis 
that included title, years of experience, size of organization and generalized geographic 
location. 

The second area. Questions 9 through 14, inquire about information system 
certification and accreditation concerns, vulnerability scanning, and patch management 
practices. Similar surveys regarding these topics were sought during secondary research 
efforts, but were not available. Therefore, the survey questions were developed from a 
review of the literature available regarding information security, current technological 
advances, and interviews with Information Security professors at the Naval Postgraduate 
School. The response options vary somewhat, but the selection of responses are limited 
to four choices ranging from Yes, No, Planned, and Don’t Know throughout this section. 

The third section consisted of Questions 15 through 24, which seeks information 
regarding known system compromises, number of personnel available to perform 
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maintenance, and average number of hours spent performing maintenance and/ or 
restoration efforts. The response options vary somewhat with numerical response options 
that are ranges of approximation. It was perceived that an exact accounting of previous 
incidents may have deterred survey participation survey. 

Section Four is reserved for Questions 25 through 30, separated because they 
yield a wide variety of response options. Further, survey respondents were cautioned 
regarding the dissimilarities in response options. This section seeks information 
regarding information asset totals, personnel strengths, and personal opinions. 

Fellow students enrolled in the information security track within the Information 
Systems Technology curriculum at the Naval Postgraduate School were selected to edit 
the survey instrument before pre-testing. Afterwards, the survey was submitted for pre¬ 
testing to the Naval Postgraduate School Chief Information Officer (CIO), Chief 
Technology Officer (CTO), Information Systems Security Officer (ISSO), and 
Information Systems Security Manager (ISSM) to average the total survey participation 
time and to solicit any noted discrepancies or potential conflicts within the survey. Their 
input had a two-fold benefit: a test of the survey instrument for readability and allowance 
for editing without eliminating potential respondents from the pool of professionals 
within Naval Medicine. 

Prior to distributing the survey, the NMO portal manager at Naval Medical 
Information Management Center, Bethesda, Maryland, was contacted to request 
permission to distribute the survey instrument. Once permission was received, potential 
respondents were contacted via the global address book on the Naval Medicine domain 
and asked to volunteer for the study. 
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IV. RESEARCH FINDING AND ANALYSIS 


A, INTRODUCTION 

The small number of Naval Medicine CIOs available for this study required 
utilization of a similar group to pre-test the survey questionnaire. The survey 
questionnaire for this study was pre-tested by fellow students enrolled in the information 
security track within the Information Systems Technology curriculum at the Naval 
Postgraduate School. Afterwards, the survey was submitted for pre-testing to the Naval 
Postgraduate School CIO, CTO, ISSO, and ISSM to average the total survey participation 
time and to solicit any noted discrepancies or potential conflicts within the survey. The 
average time to complete the 30-question survey was approximately 12 minutes. Pre-test 
questionnaires are located in Appendix B. Following the pre-test, questionnaires were 
transformed to an HTML-based survey and were activated on the Naval Medicine Online 
portal. Finally, each of the 51 identified Naval Medicine CIOs was sent a survey key via 
email with an accompanying message to explain the purpose of the survey. 

B. DEMOGRAPHIC ANALYSIS 

Naval Medicine employs approximately 60,000 military, civilian and contract 
personnel to support medical and dental facilities, health care support offices, research 
and development activities and training commands around the world. The information 
technology components of approximately 300 facilities are currently managed by 51 
CIOs. A total of 51 survey invitations were sent out for participation in this research 
effort and 31 anonymously completed surveys were posted to the database. The 
anonymous respondent survey data is located in Appendix D. Twenty-six (84 percent) of 
the survey respondents were located in the continental United States (Inconus) and five 
(16 percent) of the survey respondents were located outside of the continental United 
States. See Figure 5. This sample size represents approximately 61 percent of the CIOs 
in Naval Medicine. 
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Figure 5. Regional Survey Response 

There were 27 CIOs, three ISSMs, and one respondent categorized as “Other” that 
made up the representative sample of survey respondents. See Figure 6. The CIOs made 
up approximately 88 percent of the sample, the ISSMs made up another 9 percent of the 
population and one survey respondent was listed as “Other” for a job title that accounted 
for approximately 3 percent of the survey sample. 



Figure 6. Survey Respondent Titles 
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This researcher attempted to make a distinction in facility size by incorporating 
ranges of total personnel strength per facility to classify each as a small, medium, or large 
facility. Thirteen percent of the respondents were responsible for facilities with more 
than 1000 personnel within their organization. Fifty-five percent of the respondents were 
responsible for medium-sized facilities ranging from 201 to 1000 personnel. The other 
32 percent were responsible for smaller facilities with less than 200 personnel on board. 


y 

Figure 7. Respondent Organization Size (Personnel Strength) 

Seventy-one percent of the sample had eight or more years of IT experience. 
There were no respondents with less than 2 years of experience. See Figure 8. Only 10 
percent of the sample population had between 2 and 4 years of experience in IT 
operations. 
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Figure 8. Respondent Years of Experience 


C. SURVEY QUESTIONNAIRE ANALYSIS 

The standard deviation and confidence levels at 95% were computed for all 
questions. See Appendix F. Standard deviation remains one of the most commonly used 
statistical tools in the sciences and social sciences. It provides a precise measure of the 
amount of variation in any group of numbers. A standard deviation is the plus or minus 
variance from the mean score needed to capture 68 percent of the population. More 
generally, it is a number that distinguishes how far a particular field of data varies from 
the overall average of all respondents that answered a particular question. The smaller the 
deviation, the more confidence one can have in the computed value for the mean. An 
extreme deviation was not noted in this survey. Question 26 (“How many months pass 
between each systems vulnerability/penetration test?”) had the highest standard deviation 
at 1.362890, and Question 16 (How many system compromises were considered as 
serious?) had the lowest standard deviation at 0.358568. Those figures are represented in 
Appendix G. 

A comparison of survey questions 15 (“How many known systems compromises, 
including e-mail and Web-based deliveries of malware, have occurred within your 
organization in the past year?”) and 17 (“How many of the compromises may have been 
prevented if the available patches had been installed?”) revealed that of the 14 commands 
reporting known compromises, seven of them reported that the available vulnerability 
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patches had not been applied. Fifty pereent of the attaeks oeeurred on unpatehed 
systems. The survey did not request speeifie numbers of known attaeks; however, it 
requested a general range of known attaeks (See Appendix A, Question 15). On the 
eonservative side, that total equates to as little as 31 known attaeks and at the other end of 
the response eategory speetrum, it equates to as many as 72 known attaeks in the past 12 
months. Along with this data, it should also be noted that 77 pereent of those 
eompromised systems were at eommands within the eontinental United States (Ineonus), 
while the remaining 23 pereent were outside the eontinental United States (Outeonus). 
This finding eorrelates to the findings in Burns (2003), whereas the majority of attaeks 
are direeted at the United States. 

One would tend to believe that years of experienee would make a differenee in the 
frequeney of attaeks. The demographies portion of the survey inquired about the 
respondent’s years of IT experienee. Onee again, this area of the survey did not request 
speeifie numbers of years; however, it requested a general range of years of experienee. 
(See Appendix A, Question 4). An aeross-the-board eonservative approaeh, awarding no 
more that 8 years of IT experienee to any one person, revealed that the average IT 
experienee level for those eompromised eommands is 6.76 years. This average eould 
indieate that some of the CIOs in the field are short on either personnel or resources given 
that they have a adequate amount of IT experienee. 

Question 25 inquired about the number of personnel resourees employed 
speeifieally to perform system eonfiguration and pateh management tasks. Question 28 
asked if enough resourees were available to meet the eurrent seeurity threat. Eight (57%) 
of the 14 sites reporting eompromises reported they had the appropriate resourees. Of 
those 14 sites that experieneed system eompromises, nine (64%) of them had one or more 
personnel performing that duty. The other five sites that were eompromised did not have 
personnel assigned speeifieally to perform pateh management tasks, but five (83%) had 
someone performing those tasks as a part of their responsibility. Sinee 13 of the 14 
(93%) of the eompromised sites have someone performing the tasks in some eapaeity, 
this eould indieate that the eurrent pateh management tasks are not being performed as 
quiekly, or as thoroughly, as neeessary to prevent the eompromises as other tasks may be 
taking preeedenee. 
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Question 26 asked how many months pass between eaeh systems vulnerability 
test. The eumulative average of all eompromised commands is 6.3 months between 
systems scans, while those that were not compromised average 4.6 months between 
vulnerability scans. This supports Nicolett and Pescatore’s theory (2003) that although 
malware and bugs may take 6 months to become a problem, more frequent scanning will 
have an important effect on network management. At a minimum. Military Health 
System Information Assurance Policy (2003) recommends system vulnerability scans at 
least once per month. Gartner research predicts that by the year 2005, just 6 months from 
now, “... the due diligence level of vulnerability assessment will require that full system 
scans be done at least once per month (.07 probability).” Naval Medicine, as indicated by 
the lA survey results, is far behind the requirement for monthly scans, which if performed 
as prescribed, would have eliminated a significant amount of intrusions. Depending on 
the size of the organization, automated methods of scanning may increase effectiveness, 
while simultaneously decreasing the overall risk. 

An insight to scanning and patching practices was discovered in Questions 10 
through 13. Question 10 asked if automated vulnerability scans were performed. 
Eighteen (58%) commands reported that automated scans are done, while the remaining 
13 (42%) reported that they did not perform them. Question 11 was the follow-on to 
Question 10, as it inquired about how many commands are utilizing automated patching 
technologies. Twenty-five (81%) reported that they did, and only six (19%) reported that 
they did not. This is interesting, to say the least if when considering the number of know 
compromises over the past 12 months. Those reporting automated patching may be 
indicating that their servers are only performing automated vendor downloads and 
updates, as are available now on many of the Microsoft and Unix operating systems. 
Question 12 asked if manual vulnerability scans were performed. Nineteen (61%) 
commands reported that manual scans are done and the remaining 12 (39%) reported that 
they did not perform them. Of all respondents surveyed, 14 (45%) of them are 
performing scans within the 1-3 month timeframe, and it becomes evident at this point 
that frequency of scans and remediation has the greatest impact in regards to compromise 
prevention. Only 4 (29%) of the 14 respondents have reported having systems 
compromises within the past year. Question 13 was the follow-on to Question 12, 
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inquiring as to how many commands are utilizing manual patching technologies. 
Twenty-two (71%) reported that they did and nine (29%) reported that they did not. 
Seventeen (55%) respondents reported that automated patching and scanning was being 
performed, but collectively, they were compromised eight times. One might question the 
length of time between detection and remediation of known vulnerabilities. A closer look 
at the responses regarding automated scanning and automated patching revealed that the 
average length of time between scans is 5.3 months. Three (37.5%) of those 
organizations scan and patch every 1-3 months, 2 (25%) others scan every 4 -6 months, 
and 3 (37.5%) others scan every 10-12 months. Of those eight compromises, one (13%) 
was serious, and 3 (37.5%) were reported to be lacking the appropriate patch. 
Furthermore, 6 (75%) of those compromises originated from email and 2 (25%) were 
compromised via the Web. Another discovery was that the other 14 respondents that did 
not perform both automatic scanning and patching were collectively compromised seven 
times. The responses for both Questions 10 and 11 revealed that 50 percent of those 
commands not performing automated scanning and patching on a continual basis 
maintain 50 percent of the systems compromised over the past 12 months. The review of 
Questions 12 and 13, regarding manual scanning and patching, revealed that of the 16 
(52%) commands that utilize manual methods, collectively they contributed to 
approximately 44% of the past year’s compromised systems, as seven (44%) of the 16 
sites experienced a compromise. Interestingly, six of those compromises stemmed from 
email and one was due to an outdated anti-virus signature. The 15 (48%) respondents 
that do not utilize manual scan patching methods collectively contributed to eight systems 
compromises, which is approximately 53% of the total. Questions 10 through 13, which 
cover both automated and manual scanning and patching practices revealed that 11 (35%) 
of all survey respondents were utilizing both methods and 20 (65%) were not. The 
average time between scans for those that were utilizing both methods was 5.2 months. 
Of the 11 (35%) that were performing both, 5 (45%) had been compromised. As one 
may easily recognize, automation does not provide a significant advantage over manual 
methods if the tools are not being employed on a monthly basis as required by the 
Military Health Systems (MHS) Information Assurance Policy. This survey should 
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highlight the speed in whieh the new threats are approaching and that automation must be 
utilized on a more frequent basis. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CORRELATION OF RESULTS IN COMPARISON TO PREMISE 

The premise that 75 percent of Naval Medicine’s known information systems 
compromises were not protected by the available vulnerability patch(s) was not 
confirmed. The findings and consolidated view of this study are depicted below in 
Figure 9 as it relates to significantly disproving the premise of this project. 
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Figure 9. Regional Survey Response 

Survey respondents from 14 (45%) commands reported known systems 
compromises within the past year. Seven (50%) of those respondents reported that the 
available vulnerability patches had not been applied. In essence, 50 percent of the known 
attacks were due to the tardiness in application of the appropriate protection. It bears 
repeating that the average length of time between automated vulnerability scans 
combined with automated patching is 5.3 months. If this average detection and response 
time is not corrected, the next 12 months may show a marked increase in system 
compromises. 

B. THESIS QUESTIONS REVIEW 

I. Are existing Naval Medicine Information Assurance policies in alignment 
with current Navy policy and federal government requirements? The existing policies 
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are all vague and general with regard to vulnerability assessments and patehing 
requirements. MHS has the most up-to-date information, but the requirements within that 
doeument still fall short of best-business practices. Nearly every published document 
pertaining to information assurance within the past year highlights the current threats, 
reports that the patch management industry is a spiraling market, and that near real time 
scanning and patching are the only real options left to safeguard connected information 
assets. 

2. Would the implementation of automated vulnerability scanning and 
patching technology benefit Naval Medicine? As previously mentioned in Question 1, 
automated scanning and patching solutions as close to real time as possible are the most 
effective means to securing information assets to protect them form the current threat 
environment. The continued occasional use will not provide the true ROI associated with 
more aggressive automated vulnerability assessment practices. 

3. Would automated vulnerability scanning and patching be a cost-effective 
means to address the current information assurance threats? Gartner group estimates 
that a 1000 unit server farm costs $300,000 per year to perform patch management tasks. 
The same server population would cost $50,000 to implement an automated solution. 
Since this researcher does not have access to the actual number of systems within Naval 
Medicine, the following subjective estimate is submitted. 

BUMED estimates personnel strength to be approximately 60,000. If only 25 
percent of those personnel utilized information systems to perform their tasks, that would 
mean that Naval Medicine maintains about 15,000 systems. The sample population for 
this lA survey reports that the aggregate of respondents currently have 27 dedicated 
personnel to perform patch management within their organization. The sample size only 
represented approximately 61 percent of the CIOs in Naval Medicine. If all had 
responded, the results may have approached a total of approximately 50 personnel that 
performed patch management as their primary duty. If each of them maintains a salary 
of $40,000 per year. Naval Medicine spends approximately $2M per year in maintenance 
costs to protect their information assets, while 50 percent of manually scanned and 
patched systems are compromised within a one-year time frame. If these figures are 
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somewhat close to the truth, a commercial vendor's automated scanning and patching 
solution would cost approximately $750,000 per year. 

In addition, one must consider that the conservative approach of 31 known 
reported system compromises as reported by the survey respondents occurred within the 
past year. It is unknown which systems were compromised, but if personal privacy data 
was compromised on any of them, the fiscal penalties from the HIPAA violations alone 
could easily go beyond $75 OK. 

Outside of the HIPAA requirements, the reported costs to rebuild compromised 
systems takes approximately 2-4 hours depending on the operating system and data files 
required. Currently 17 of the respondents report that they spend 10 + hours per system 
per month to keep each system patched and configured to meet the current information 
assurance threats. Considering that a system administrator’s salary is approximately 
$40,000 per year, each month’s maintenance per system is approximately $3,300 dollars. 
If that figure is divided in half, the cost remains approximately $1,500 per month per 
system for the required maintenance. If the vendor solution is $50 per year per system, 
recurring maintenance fees based on $1,500 per month cost approximately $18,000 per 
year. That fee of $18,000 divided by $50.00 represents a 180 percent savings in 
maintenance costs if only half of the administrators were utilized. This automation would 
not replace the administrative staff, but it would free up their valuable time to work on 
other significant maintenance issues. The automated technology is well worth what it 
provides, but Naval Medicine personnel may not have the funding to invest in these 
technologies right away; however, options for automated solutions are free of charge 
from DISA and the Department of Homeland Security. “An effective vulnerability- 
assessment/patch management effort will reduce operational risks for everyone” (Shipley, 
2003). 

4. Would a consolidated and centrally managed vulnerability database 
increase the current security posture? 

Centrally managed vulnerability databases are already maintained by federally 
funded organizations such as NIST. In the case of a centrally managed database that 
maintains tested patches, it may be helpful to have a secure source to pull from, but many 
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are opting out of patch testing as they would rather take one of their own systems offline 
as opposed to having their entire network, in many cases, exposed to the malware 
practitioners and hackers looking for free spam relays (Roberts, 2003). 

C. RECOMMENDATIONS FOR INCREASING NAVAL MEDICINE 

INFOSEC POSTURE 

The lA survey revealed that approximately 42% of the respondents are not 
utilizing automated vulnerability assessment tools. This translates to increased risk, 
increased costs and a lower confidence level for those personnel charged with the 
responsibility of safeguarding the organizations information assets. Automation has 
faced much criticism in the past, as has any new technology. According to the lA survey, 
45% of the respondents have concerns regarding reliability and another 32% are 
concerned about the effectiveness of automated assessment and patching solutions. This 
researcher submits that nothing will ever be bulletproof, but proactively utilizing the best 
tools available to offset the threat will always remain the best defense (Shipley, January 
2003). 

The perceived benefit derived from the utilization of automated vulnerability 
assessment solutions can only promote a healthier and more secure networking 
environment for Naval Medicine professionals, while significantly decreasing the overall 
risk (Shipley, January 2003). The continued increases of malware distribution, in 
conjunction with the increased reliance on networked information systems, create an 
overwhelming need to maintain confidentiality, integrity, and availability of information 
assets. The personnel hours required in typical monthly maintenance procedures alone 
will produce an immediate return on investment if funding is unavailable for an 
enterprise-wide solution (Schroder, 2003). If funding is not available, an immediate 
effort should be made by those commands not using automated solutions to obtain the 
free GOTS vulnerability scanning solution from DISA. 

Security experts around the globe concur that today’s networked environment is 
more dangerous than it has ever been and those that do not utilize automated solutions in 
conjunction with layered defenses are at a much greater risk than those that are taking 
more proactive and aggressive approaches to securing their information assets. 
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D, 


SUGGESTIONS FOR FURTHER RESEARCH 


A comparison of blocked attacks per organization in comparison to vulnerability 
scanning and mitigation practices may yield even more evidence regarding the 
probability of enhaneed security based on speed in detection and mitigation of risk. It 
would also be interesting to know how many eompromises have been avoided within 
Naval Medieine due to the use of automated or manual vulnerability assessment methods 
and whieh tools were eonsidered the best across the boards (i.e., ease of use, lieensing 
expenses, etc.). In addition, a comparison of formal policies and praetices among those 
commands and regions may also provide for interesting researeh. If BUMED began 
keeping traek of eompromises, over time, deviations in command practices and policy 
adherence may beeome more evident. This survey only asked for the past 12 months of 
history. Another year or two of analysis would have been highly beneficial to this 
research project. 
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APPENDIX A. SAMPLE HTML RESEARCH QUESTIONNAIRE 


Naval Medicine Online 

Information Assurance Management Survey 

As Navy Medicine IT leaders dealing with dynamic and complex 
environments, I am respectfully requesting your assistance In completing 
my thesis research at Naval Postgraduate School. 

The most recent trends In Information Assurance have Illustrated the 
worsening of vulnerabilities and exploits. This trend Is easily Identified In 
historical CERT summary reports. My thesis topic focuses on Identifying 
more manageable methods to conduct vulnerability scans and patch 
management tasks. This study Is resultant of my past experiences with our 
seemingly ceaseless efforts to keep NMIMC's systems scanned and patched 
by utilizing manual methods. 

As part of my research, I am posting this four-section Information 
Assurance Management Survey totaling 30 questions that should take 
approximately 10-15 minutes to complete. As noted on the survey form, 
your Input will remain completely anonymous for obvious reasons. Your 
assistance In completing this survey no later than April 30, 2004 would be 
greatly appreciated. Your Input will make a difference. A single use survey 
key has been provided to you to ensure each command responds only once. 

This survey Instrument has been validated by a number of professors within 
the Center for Information Systems Security Studies and Research (CISR) 
at NPS and tested among local professionals managing the NPS networks. 
The final review was performed by Dr. Dorothy Denning, a well respected 
Information security expert. 

I thank you In advance for your consideration and participation In this 
study. If you are Interested In obtaining a copy of the results, please send 
an e-mall request to (sprelnke@nps.navy.mil) at your earliest convenience. 

Very Respectfully, 

LTjg Steven Reinkemeyer 
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* Answer Required. 

Section 1 - Demographics 

Please choose one response per question. 

1. What is your title? * 

CIO 

ISSM 

ISSO 

Other 

2. How many years of IT experience do you have? * 

0-1 

2-3 

4-5 

6-7 

8 + 

3. Where is your geographic location? * 

INCONUS 

OUTCONUS 

4. How many personnel are employed within your organization? * 

< 200 
201 - 1000 
> 1000 
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Section II 

Please choose one response for each question. 

5. Are all of your organization's applications certified and accredited 
under a full Authority to Operate (ATO)? * 

Yes 
C No 
r^. Planned 
C Don't Know 

6. Are all of your organization's servers certified and accredited 
under a full Authority to Operate (ATO)? * 

n Yes 
No 

Planned 
C Don't Know 

7. Is your organization's network certified and accredited under a 
full Authority to Operate (ATO)? * 

w Yes 
C No 
C Planned 
C Don't Know 

8. Does your organization have a written vulnerability assessment 
policy (e.g., maximum amount of time between assessments)? * 

f' Yes 
.. No 
C Planned 
C Don't Know 

9. Does your organization have a written patch management policy 
(e.g., patch prioritization based on risk or threat)? * 

C Yes 
u. No 
< Planned 
C Don't Know 
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10. Does your organization perform automatecl vulnerability 
assessments? * 

C Yes 
No 

C Planned 
C Don't Know 

11. Does your organization use automated patch management 
tools? * 

-j Yes 
r No 
C Planned 
C Don't Know 

12. Does your organization perform manual vulnerability 
assessments? * 

Yes 
C No 
C Planned 
C Don't Know 

13. Does your organization apply patches manually? * 

^ Yes 
C No 
C Planned 
C Don't Know 

14. Does your organization use stand-alone systems to test patches 
before applying them to affected systems? 

C Yes 
C No 
C Planned 
C Don't Know 
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Section III 

The response options for questions 15-24 are approximations. Please 
choose one response per question. 

15. How many known system compromises (e.g., unauthorized 
system events or data theft) including e-mail and Web-based 
deliveries of malware have occurred within your organization in the 
past year? 

If you select 0, skip to question 23. 

C 0 
^ 1-4 
5-9 

C 10 -I- 

16. How many of those compromises were considered as serious 
(e.g., great effort to restore, many systems affected, or higher 
authority intervention)? 

C 0 
r 1-4 
5-9 

C 10 + 

17. How many of the compromises may have been prevented if the 
available patches had been installed? 

C 0 
^ 1-4 
C 5-9 
C 10 -I- 

18. How many known system compromises (e.g., unauthorized 
system events or data theft) were from e-mail de^livered malware 
(e.g., worms, viruses, Trojans, etc.) in the past year? 

** If you select 0, skip to question 21. 

C 0 
C 1-4 
C 5-9 
C 10 -I- 
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19. How many of the e-mail delivered compromises were 
considered as serious (e.g., great effort to restore, many systems 
affected, or higher authority intervention)? 

C 0 
C 1-4 
C 5-9 
C 10 + 

20. How many of the e-mail delivered compromises may have been 
prevented if anti-virus signatures had been up to date? 

C 0 
C 1-4 
C 5-9 
C 10 -t- 

21. How many known system compromises (e.g., unauthorized 
system events or data theft) were from Web-bas^_malware (e.g., 
worms, viruses, Trojans, etc.) in the past year? 

U 0 
C 1-4 
C 5-9 
C 10 H- 

22. What is the average number of hours spent per month to 
decontaminate or remediate EACH system compromise within your 
organization? 

C 0 
C 1-4 
C 5-9 
C 10 -t- 
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23. What is the average number of hours spent per month to keep 
EACH system patched and configured to meet new security threats? 


C 0 
C 1-4 
C 5-9 
C 10 -I- 

24. How many system administrators does your organization 
employ to perform patch management/system configuration tasks 
that are incorporated with their other responsibilities? * 

C 0 
C 1-4 
C 5-9 
^ 10 -t- 

Section IV 

Questions 25-30 have a variety of responses. Please read carefully and 
choose one response per question. 

25. How many system administrators does your organization 
employ to perform patch management/system configuration tasks 
on your network as their primary duty? * 

C 0 
C 1 
C 2 
C 3+ 

26. How many months pass between each systems 
vulnerability/penetration test? * 

C 1-3 
C 4-6 
U 7-9 
C 10-12 -I- 
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27. Approximately how many servers reside on your network? * 

0-50 
51-100 
C 101-150 
> 150 

28. Do you believe you have sufficient resources to keep each 
system patched and configured to meet new security threats? * 

o Yes 
C No 
C Planned 
C Don't Know 

29. What is your greatest concern with using automated 
vulnerability/patch management tools? * 

Effectiveness 
C Reliability 
C Compatibility 
f Other 

30. What type of assistance from the DoD would most greatly assist 
you in your lA efforts? * 

I Training 
^ Funding 
C Tools 
Services 

* Answer Required. 

Submit Survey 
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APPENDIX B. SURVEY PRE-TEST QUESTIONNAIRES 


Information Assurance Management Survey 

This research is being conducted at the Naval Postgraduate School in partial fulfillment of the requirements lor a 
Masters of Science degree in Information Systems Technology. 

** Your answers will remain anonymous. 


Please choose one response in each category 


1 Job Title 1 CIO 

I ISSM 1 ISSO 

1 Other 1 


1 Years of Experience 1 

0-1 1-2-3 1 4-5 

T 6-7 1 8+ 1 


1 Geographic Location 

1 INCONUS'1 

OUTCONUS 1 


I Organization Size T< 200 personiiel I 201 -1000 personnel I > 1000 personnel | 


1 

_^- 

Are all of uoitr ornanization’s aoolications certified 

and accredited under a full Authority to Operate 
(AT01? 

Yes 

No 

Planned 

Don't 

Know 

2 

Am all of your oroanizalion's servers certified and 

accredited under a full Authority to Operate (ATO)? 

Yes 

No 

Planned 

Don't 

Know 

3 

Is your nrnanization's network certified and 

accredited under a full Authority to Operate (ATO)? 


No 

Planned 

Don't 

Know 

4 

Doos your organization have a written vulnerability 

assessment policy (e.g., maximum amount of time 
between assessments)? 


No 

Planned 

Don't 

Know 

5 

Does your organization have a written patch 

management policy (e.g., prioritization based on risk 
or threat)? 

'<&&J 

No 

Planned 

Don't 

Know 

6 

nrjps ynnr nr^enizetion oerform automated 

vulnerability assessments? 

Yes 

No 

PlannedH 

Don't 

Know 

7 

Does your oraanization use automated patch 

management tools? 

Ye^ 

No 

Planned 

Don't 

Know 

8 

Does your organization perform manual vulnerability 

assessments? 

'^Yes 

No 

Planned 

Don't 

Know 

9 

Does your organization apply patches manually? 

[yTs 

No ' 

Planned 

Don't 

Know 

10 

Does your organization use stand-alone systems to 
test patches before applying them to affected 

1 systems? 


No 

Planned 

Don't 

Know 


The response options for questions 11-19 are approximations. 

> ‘'■‘r- ___ - 

^ I . - I I e a CO r- m . 


-1 

11 

How many known system compromises (e.g., 
unauthorized system events or data theft) have 
occurred within your organization in the past year? 
ff you sefecf 0, siao to aueatlon 18. 

0 

1-4 

5-9 

^ 10-1- 

12 

How many of those compromises were considered 

as serious (e.g.. great effort to restore, many 
systems affected, or higher authority intervention)? 


1-4 

5-9 

10-f 
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13 

How many of the compromises may have been 
prevented if the avail^le patches had been 
installed? 

0 

1-4 

5-9 

1 10-t; 

14 

How many known system compromises (i.e., 

unauthorized system events or data theft) were from 
email delivered malware (e.g., worms, viruses, 
Trojans, etc.) in the past year? If you select 0, skip 

toauoatlon 17. 

0 

1-4 

5-9 

/ 10+i 

15 

How many of the email delivered compromises were 

considered as serious (e.g., great effort to restore, 
many systems affected, or higher authority 
intervention)? 


1-4 

5-9 

10 + 

16 

How many of the email delivered compromises may 

have been prevented if anti-virus signatures had 
been up to date? 

0 

1-4 

5-9 

10 +i 

17 

What is the averaoe number of hours spent per 

month to de^ntaminate or remediate each system 
comprontiseWhin vour oroanization? 

0 

1-4j 

5-9 

10 + 

18 

What is the averaoe number of hours snoot per 

month to keep each system patched and configured 
to meet new security threats? 

0 


5-9 

• 

V llo-t 

19 

How many system administrators does your 

organization employ to perform patch 
management/system configuration tasks that are 
incorporated with their other responsibilities? 

0 

1-4^, 

5-9 

10 + 


A'' 


The remaining questions have a variety of responses. Please read carefully. 


20 

How many system administrators does your 
organization employ to perform patch 
management/system configuration tasks on your 
networkfs) as their orimarv dutv? 

0 

1 


3 + 

21 

How many months pass between each systems’ 
vulnerability/penetration test? 

1-3 

4-6 

7-9 

- \ 

10-12 + 

22 

Approximately how many servers reside on your 

network? 

0-50 

51-100 

: 101-150 ’ 

> 150 

23 

Do you believe you have sufficient resources to keep 

each system patched and configured to meet new 
security threats? 

Yes 

"x 

Planned 

N/A 

24 

What is your greatest concern with using automated 

vulnerability/patch management tools? 

Effeclivefi 

ness L, 

Reltabiitty 

Compat)' 
bllity / 

None 

25 

What type of assistance from the DoD would most 

greatly assist you in your lA efforts? 

Training 


Tools 

Senrices 
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Information Assurance Management Survey 

This research is being conducted at the Naval Postgraduate School in partial fulfillment of the requirements for a 
Masters of Science degree in Information Systems Technology 

** Your answers will remain anonymous. 

Please choose one response in each cateoorv 


I Job TKIo I CIO 

I XISSM j ISSO 

1 Other 1 


I Years of Experience | 

0-1 1 2-3 1 4-5 

1 X6-7 1 8+ 1 


{ Geographic Location 

1 XINCONUS 1 

OUTCONUS 1 


I Organiza tion Sire j < 200 personnel | 201 - 1000 personnel | X> 1000 perso nnel | 


1 

Are all of your oroanization's apolications certified 
and accredited under a full Authority to Operate 
(ATO)? 

Yes 

No 

XPIanned 

Don't 

Know 

2 

Are all of your oroanization’s servers certified and 

accredited under a full Authority to Operate (ATO)? 

Yes 

No 

XPlanned 

Don't 

Know 

3 

Is your organization's network certified and 

accredited under a full Authority to Operate (ATO)? 

Yes 

No 

XPIanned 

Don't 

Know 

4 

Does your organization have a written vulnerability 
assessment policy (e.g., maximum amount of time 
between assessments)? 

XYes 

No 

Planned 

Don't 

Know 

5 

Does your organization have a wntten patch 

management policy (e g., priohtization based on risk 
or threat)? 

XYes 

No 

Planned 

Don't 

Know 

6 

Does your organization perfonn automated 

vulnerability assessments? 

XYes 

No 

Planned 

Don't 

Know 

7 

Does your organization use automated oatch 
management tools? 

xYes 

No 

Planned 

Don't 

Know 

8 

Does your organization perform manual vulnerability 

assessments? 

xYes 

No 

Planned 

Don’t 

Know 

9 

Does your organization apply patches manually? 

xYes 

No 

Planned 

Don't 

Know 

10 

Does your organization use stand-alone systems to 
test p^ches before applying them to affected 
systems? 

xYes 

No 

Planned 

Don't 

Know 


The response options for questions 11-19 are approximations. 


11 

How many known system compromises (e g., 

unauthorized system events or data theft) have 
occurred within your organization In the past year? 

If you select 0, skip to Question 18. 

xO 

1-4 

5-9 

10 + 

12 

How many of those compromises were considered 

as serious (e.g., great effort to restore, many 
systems affected, or higher authority intervention)? 

xO 

1-4 

5-9 

10 + 
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13 

How many of the compromises may have been 
prevented if the available patches had been 
installed? 

0 

1-4 

5-9 

x10 + 

14 

How many known system compromises (i.e., 
unauthorized system events or data theft) were from 
email delivered malware (e g., worms, viruses. 

Trojans, etc.) in the past year? If you select 0, skip 
to Question 17. 

0 

1-4 

5-9 

x10 + 

15 

How many of the email delivered compromises were 
considered as serious (e.g., great effort to restore, 
many systems affected, or higher authority 
intervention)? 

xO 

1-4 

5-9 

10 + 

16 

How many of the email delivered compromises may 
have been prevented if anti-virus signatures had 
been up to date? 

0 

x1-4 

5-9 

10 + 

17 

What is the averaoe number of hours soent per 
month to decontaminate or remediate each system 
compromise within your organization? 

0 

1-4 

5-9 

x10 + 

18 

What is the averaoe number of hours spent per 
month to keep each system patched and configured 
to meet new security threats? 

0 

1-4 

5-9 

x10 + 

19 

How many system administrators does your 
organization employ to perform patch 
management/system configuration tasks that are 
incorporated with their other responsibilities? 

0 

1-4 

5-9 

x10 + 


The remaining questions have a variety of responses. Piease read carefully. 


20 

How many system administrators does your 
organization employ to perform patch 
management/system configuration tasks on your 
networfc(s) as their orlmarv duty? 

0 

1 

x2 

3 + 

21 

How many months pass between each systems' 
vulnerability/penetration test? 

x1-3 

4-6 

7-9 

10-12 + 

22 

Approximately how many servers reside on your 
network? 

0-50 

51-100 

xlOI-150 

> 150 

23 

Do you believe you have sufficient resources to keep 
ear^ system patched and configured to meet new 
security threats? 

Yes 

xNo 

Planned 

N/A 

24 

What is your greatest concern with using automated 
vulnerability/patch management tools? 

Effective¬ 

ness 

xReMaMIt 

y 

Compati¬ 

bility 

None 

25 

What type of assistance from the DoD would most 
greatly assist you in your lA efforts? 

Training 

Funding 

xTools 

Sen/lces 
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APPENDIX C. ENTIRE POPULATION - RAW DATA SURVEY 

RESPONSE SPREADSHEET 


1. 

2. 

3. 

4. 

5. 

What Is your job 
title? 

How many years of IT 
experience do you 
have? 

Where is your 
geographic location? 

How many personnei are 
empioyed within your 
organization? 

Are aii of your organization's 
appiications certified and 
accredited under a fuli 

Authority to Operate (ATO)? 

Other 

8+ 

INCONUS 

<200 

No 

CIO 

8+ 

INCONUS 

201 - 1000 

No 

CIO 

2-3 

INCONUS 

<200 

Yes 

CIO 

8+ 

INCONUS 

201 - 1000 

Planned 

ISSM 

8+ 

INCONUS 

<200 

Planned 

CIO 

6-7 

INCONUS 

201 - 1000 

Planned 

CIO 

8+ 

INCONUS 

201 - 1000 

Yes 

CIO 

8+ 

INCONUS 

<200 

No 

CIO 

4-5 

OUTCONUS 

<200 

Don't Know 

ISSM 

8+ 

INCONUS 

201 - 1000 

Don't Know 

CIO 

2-3 

INCONUS 

201 - 1000 

No 

CIO 

8+ 

INCONUS 

<200 

No 

CIO 

8+ 

OUTCONUS 

201 - 1000 

Yes 

CIO 

8+ 

INCONUS 

<200 

Yes 

CIO 

8+ 

INCONUS 

201 - 1000 

Planned 

CIO 

2-3 

INCONUS 

201 - 1000 

Yes 

CIO 

8+ 

INCONUS 

201 - 1000 

Don't Know 

CIO 

4-5 

OUTCONUS 

201 - 1000 

No 

CIO 

4-5 

INCONUS 

<200 

Don't Know 

CIO 

8+ 

INCONUS 

201 - 1000 

No 

ISSM 

8+ 

INCONUS 

201 - 1000 

Yes 

CIO 

8+ 

INCONUS 

201 - 1000 

Planned 

CIO 

8+ 

INCONUS 

> 1000 

Planned 

CIO 

8+ 

INCONUS 

> 1000 

Planned 

CIO 

8+ 

INCONUS 

> 1000 

No 

CIO 

8+ 

INCONUS 

201 - 1000 

No 

CIO 

8+ 

OUTCONUS 

201 - 1000 

Planned 

CIO 

6-7 

INCONUS 

201 - 1000 

No 

CIO 

4-5 

INCONUS 

> 1000 

No 

CIO 

8+ 

INCONUS 

<200 

Planned 

CIO 

8+ 

OUTCONUS 

<200 

No 
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6. 

7. 

8. 

9. 

Are all of your organization's 
servers certified and 
accredited under a fuii 

Authority to Operate (ATO)? 

Is your organization's network 
certified and accredited under 
a fuii Authority to Operate 
(ATO)? 

Does your organization have a 
written vuinerabiiity 
assessment policy (e.g., 
maximum amount of time 
between assessments)? 

Does your organization have a 
written patch management 
policy (e.g., patch prioritization 
based on risk or threat)? 

No 

No 

No 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Planned 

Yes 

Planned 

Planned 

Planned 

No 

Planned 

Planned 

Yes 

Yes 

Yes 

Planned 

Planned 

Yes 

Yes 

Yes 

No 

No 

No 

No 

Yes 

Yes 

No 

Don't Know 

Yes 

No 

Don’t Know 

Don't Know 

Don't Know 

Yes 

No 

No 

No 

Yes 

Yes 

Yes 

Yes 

Planned 

Yes 

Yes 

Planned 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Planned 

Yes 

Yes 

Planned 

No 

No 

Don’t Know 

Don't Know 

Don't Know 

No 

No 

No 

Yes 

No 

Don’t Know 

Don't Know 

Don't Know 

Don’t Know 

No 

No 

No 

No 

Yes 

Yes 

Yes 

Yes 

Planned 

Planned 

Yes 

Yes 

Planned 

Planned 

No 

Yes 

Planned 

Planned 

Yes 

No 

No 

Planned 

No 

No 

No 

No 

No 

No 

No 

No 

Planned 

Yes 

Yes 

No 

No 

No 

No 

No 

Yes 

No 

Planned 

Planned 

Planned 

Yes 

Planned 

Planned 

Planned 

Planned 
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10. 

11. 

12. 

13. 

14. 

Does your organization 
perform automated 
vuinerabiiity 
assessments? 

Does your organization 
use automated 
patch management 
toois? 

Does your 

organization 

perform 

manuai 

vuinerabiiity 

assessments? 

Does your organization 
apply patches 
manually? 

Does your organization 
use stand-alone 
systems to test 
patches before 
applying them to 
affected systems? 

No 

No 

No 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Don’t Know 

Planned 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

No 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

Yes 

No 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

No 

Yes 

Yes 

Don’t Know 

Yes 

Don’t Know 

Yes 

No 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

No 

No 

No 

Yes 

Yes 

No 

No 

No 

Yes 

Yes 

Yes 

Yes 

No 

Don’t Know 

Don't Know 

Don’t Know 

Don’t Know 

Don’t Know 

No 

No 

Yes 

Yes 

No 

No 

Yes 

No 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

No 

No 

No 

Yes 

Yes 

No 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Planned 

No 

Yes 

No 

No 

No 

No 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

No 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Planned 

No 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 
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15. 

How many known system 
compromises (e.g., 
unauthorized system 
events or data theft) 
including e-mail and Web- 
based deliveries of 
malware have occurred 
within your organization in 
the past year? 

** If you select 0, skip to 
question 23. 

16. 

How many of those 

compromises were 
considered as serious 
(e.g., great effort to 
restore, many systems 
affected, or higher 
authority intervention)? 

17. 

How many of the 

compromises may have 
been prevented if the 
available patches had 
been installed? 

18. 

How many known system 
compromises (e.g., 
unauthorized system 
events or data theft) 
were from e-mail 
delivered malware (e.g., 
worms, viruses, Trojans, 
etc.) in the past year? 

** If you select 0, skip to 
question 21. 

1-4 

1-4 

1-4 

0 

0 




0 




0 




1-4 

0 

0 

0 

10 + 

0 

1-4 

10 + 

0 

0 

0 

1-4 

1-4 

0 

1-4 

0 

0 




0 

0 

0 

0 

1-4 

0 

0 

1-4 

0 




5-9 

1-4 

1-4 

1-4 

0 




0 




1-4 

0 

1-4 

1-4 

1-4 

1-4 

0 

1-4 

1-4 

0 

0 

1-4 

0 




5-9 

0 

0 

5-9 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 




1-4 

1-4 

1-4 

1-4 

0 

0 

0 


1-4 

0 

0 

1-4 

1-4 

0 

0 

1-4 

0 

0 

0 

0 

1-4 

0 

1-4 

1-4 







































































19. 20. 21. 22. 

How many of the e-mail How many of the e-mail How many known system What is the average number of 

delivered compromises delivered compromises compromises (e.g., hours spent per month 

were considered as may have been unauthorized system to decontaminate or 

serious (e.g., great effort prevented if anti-virus events or data theft) remediate EACH system 

to restore, many systems signatures had been up were from Web-based compromise within your 

affected, or higher to date? malware (e.g., worms, organization? 

authority intervention)? viruses, Trojans, etc.) in 

the past year? 



0 0 1-4 1-4 

1-4 0 0 1-4 


0 1-4 

0 1-4 0 0 
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24 . 


25 . 


26 . 


What is the average number of How many system 


hours spent per month 
to keep EACH system 
patched and configured 
to meet new security 
threats? 


administrators does your 
organization empioy to 
perform patch 
management/system 
configuration tasks that 
are incorporated with 
their other 
responsibilities? 


How many system 

administrators does your 
organization employ to 
perform patch 
management/system 
configuration tasks on 
your network as their 
primary duty? 


How many months pass 

between each systems 

vulnerability/penetration 

test? 
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27. 

28. 

29. 

30. 

Approximately how many 
servers reside on your 
network? 

Do you beiieve you have 
sufficient resources to 
keep each system 
patched and configured 
to meet new security 
threats? 

What is your greatest concern 
with using automated 
vuinerability/patch 
management toois? 

What type of assistance from 
the DoD would most 
greatly assist you in your 
lA efforts? 

0-50 

No 

Other 

Funding 

0-50 

No 

Reliability 

Tools 

0-50 

No 

Compatibility 

Funding 

0-50 

No 

Reliability 

Funding 

0-50 

Yes 

Reliability 

Training 

0-50 

Yes 

Reliability 

Funding 

0-50 

Yes 

Reliability 

Training 

0-50 

Yes 

Other 

Funding 

0-50 

No 

Effectiveness 

Services 

0-50 

Yes 

Effectiveness 

Training 

51-100 

No 

Reliability 

Funding 

0-50 

No 

Effectiveness 

Training 

0-50 

No 

Reliability 

Tools 

0-50 

Yes 

Effectiveness 

Tools 

0-50 

Yes 

Compatibility 

Funding 

0-50 

No 

Reliability 

Funding 

0-50 

Yes 

Effectiveness 

Training 

0-50 

Yes 

Reliability 

Tools 

0-50 

Don't Know 

Other 

Training 

0-50 

No 

Effectiveness 

Tools 

101-150 

Yes 

Effectiveness 

Training 

0-50 

Yes 

Reliability 

Funding 

51-100 

Yes 

Effectiveness 

Tools 

101-150 

No 

Reliability 

Funding 

0-50 

No 

Effectiveness 

Funding 

0-50 

No 

Reliability 

Training 

0-50 

Yes 

Effectiveness 

Tools 

0-50 

Yes 

Reliability 

Tools 

101-150 

No 

Other 

Tools 

0-50 

No 

Compatibility 

Tools 

0-50 

Yes 

Reliability 

Tools 
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APPENDIX D. ENTIRE POPULATION - END ANCHORED DATA 
CODING SPREADSHEET FOR QUESTIONS 5 THROUGH 30 





biiiiiiiiiiiiiiiiiiiiiiiiiiiiii 


II 















m 


m 

'vO 


00 

C^ 

O 



m 


m 


r- 

00 

C^ 




I' 


'f' 



I' 

I' 

I' 


I' 


I' 

I' 


I' 

I' 











C/!! 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 

C/D 


93 


Svy31 

Svy31 
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APPENDIX E. ALL RESPONSE STATISTICS SPREADSHEET FOR 

QUESTIONS 5 THROUGH 30 


Q#5 


Q#6 


Q#7 








Mean 

2.37931 

Mean 

2.066667 

Mean 

2.333333 

Standard Error 

0.181766 

Standard Error 

0.185282 

Standard Error 

0.187747 

Median 

2 

Median 

2 

Median 

2 

Mode 

2 

Mode 

1 

Mode 

3 

Standard Devia 

0.97884 

Standard Devia 

1.014833 

Standard Devia 

1.028334 

Sample Varianc 

0.958128 

Sample Varianc 

1.029885 

Sample Varianc 

1.057471 

Kurtosis 

-0.8984 

Kurtosis 

-0.87913 

Kurtosis 

-1.12871 

Skewness 

0.118182 

Skewness 

0.49552 

Skewness 

0.075501 

Range 

3 

Range 

3 

Range 

3 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

4 

Maximum 

4 

Sum 

69 

Sum 

62 

Sum 

70 

Count 

29 

Count 

30 

Count 

30 

Confidence Lev 

0.372331 

Confidence Lev 

0.378945 

Confidence Lev 

0.383987 


Q#8 

Q#9 

Q#10 







Mean 

2.333333 

Mean 

1.62069 

Mean 

1.5 

Standard Error 

0.187747 

Standard Error 

0.135132 

Standard Error 

0.133477 

Median 

2 

Median 

2 

Median 

1 

Mode 

3 

Mode 

1 

Mode 

1 

Standard Devia 

1.028334 

Standard Devia 

0.727706 

Standard Devia 

0.731083 

Sample Varianc 

1.057471 

Sample Varianc 

0.529557 

Sample Varianc 

0.534483 

Kurtosis 

-1.12871 

Kurtosis 

2.649576 

Kurtosis 

3.474654 

Skewness 

0.075501 

Skewness 

1.339382 

Skewness 

1.701912 

Range 

3 

Range 

3 

Range 

3 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

4 

Maximum 

4 

Sum 

70 

Sum 

47 

Sum 

45 

Count 

30 

Count 

29 

Count 

30 

Confidence Lev 

0.383987 

Confidence Lev 

0.276805 

Confidence Lev 

0.272991 


Q#11 

Q#12 

Q#13 







Mean 

1.266667 

Mean 

1.5 

Mean 

1.366667 

Standard Error 

0.126249 

Standard Error 

0.149712 

Standard Error 

0.122083 

Median 

1 

Median 

1 

Median 

1 

Mode 

1 

Mode 

1 

Mode 

1 

Standard Devia 

0.691492 

Standard Devia 

0.820008 

Standard Devia 

0.668675 

Sample Varianc 

0.478161 

Sample Varianc 

0.672414 

Sample Varianc 

0.447126 

Kurtosis 

8.877688 

Kurtosis 

4.156476 

Kurtosis 

7.219289 

Skewness 

2.942952 

Skewness 

2.010164 

Skewness 

2.37972 

Range 

3 

Range 

3 

Range 

3 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

4 

Maximum 

4 

Sum 

38 

Sum 

45 

Sum 

41 

Count 

30 

Count 

30 

Count 

30 

Confidence Lev 

0.258207 

Confidence Lev 

0.306196 

Confidence Lev 

0.249688 
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Q#14 

Q#15 

Q#16 







Mean 

1.966667 

Mean 

1.566667 

Mean 

1.142857 

Standard Error 

0.155241 

Standard Error 

0.141286 

Standard Error 

0.078246 

Median 

2 

Median 

1 

Median 

1 

Mode 

2 

Mode 

1 

Mode 

1 

Standard Devia 

0.850287 

Standard Devia 

0.773854 

Standard Devia 

0.358569 

Sample Varianc 

0.722989 

Sample Varianc 

0.598851 

Sample Varianc 

0.128571 

Kurtosis 

1.483672 

Kurtosis 

2.057227 

Kurtosis 

3.138402 

Skewness 

1.14776 

Skewness 

1.436444 

Skewness 

2.201737 

Range 

3 

Range 

3 

Range 

1 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

4 

Maximum 

2 

Sum 

59 

Sum 

47 

Sum 

24 

Count 

30 

Count 

30 

Count 

21 

Confidence Lev 

0.317503 

Confidence Lev 

0.288962 

Confidence Lev 

0.163218 


Q#17 

Q#18 

Q#19 







Mean 

1.285714 

Mean 

1.75 

Mean 

1.294118 

Standard Error 

0.101015 

Standard Error 

0.175844 

Standard Error 

0.113911 

Median 

1 

Median 

2 

Median 

1 

Mode 

1 

Mode 

2 

Mode 

1 

Standard Devia 

0.46291 

Standard Devia 

0.786398 

Standard Devia 

0.469668 

Sample Varianc 

0.214286 

Sample Varianc 

0.618421 

Sample Varianc 

0.220588 

Kurtosis 

-1.06433 

Kurtosis 

2.248449 

Kurtosis 

-1.16571 

Skewness 

1.023275 

Skewness 

1.21751 

Skewness 

0.993609 

Range 

1 

Range 

3 

Range 

1 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

2 

Maximum 

4 

Maximum 

2 

Sum 

27 

Sum 

35 

Sum 

22 

Count 

21 

Count 

20 

Count 

17 

Confidence Lev 

0.210714 

Confidence Lev 

0.368045 

Confidence Lev 

0.241481 


Q#20 

Q#21 

Q#22 







Mean 

1.294118 

Mean 

1.15 

Mean 

1.7 

Standard Error 

0.113911 

Standard Error 

0.081918 

Standard Error 

0.163836 

Median 

1 

Median 

1 

Median 

2 

Mode 

1 

Mode 

1 

Mode 

2 

Standard Devia 

0.469668 

Standard Devia 

0.366348 

Standard Devia 

0.732695 

Sample Varianc 

0.220588 

Sample Varianc 

0.134211 

Sample Varianc 

0.536842 

Kurtosis 

-1.16571 

Kurtosis 

2.775855 

Kurtosis 

3.979013 

Skewness 

0.993609 

Skewness 

2.12306 

Skewness 

1.445108 

Range 

1 

Range 

1 

Range 

3 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

2 

Maximum 

2 

Maximum 

4 

Sum 

22 

Sum 

23 

Sum 

34 

Count 

17 

Count 

20 

Count 

20 

Confidence Lev 

0.241481 

Confidence Lev 

0.171456 

Confidence Lev 

0.342912 
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Q#23 

Q#24 

Q#25 







Mean 

3.133333 

Mean 

2 

Mean 

1.933333 

Standard Error 

0.201907 

Standard Error 

0.067806 

Standard Error 

0.165629 

Median 

4 

Median 

2 

Median 

2 

Mode 

4 

Mode 

2 

Mode 

2 

Standard Devia 

1.105888 

Standard Devia 

0.371391 

Standard Devia 

0.907187 

Sample Varianc 

1.222989 

Sample Varianc 

0.137931 

Sample Varianc 

0.822989 

Kurtosis 

-1.40655 

Kurtosis 

5.581349 

Kurtosis 

0.727628 

Skewness 

-0.60801 

Skewness 

0 

Skewness 

1.028411 

Range 

3 

Range 

2 

Range 

3 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

3 

Maximum 

4 

Sum 

94 

Sum 

60 

Sum 

58 

Count 

30 

Count 

30 

Count 

30 

Confidence Lev 

0.412946 

Confidence Lev 

0.13868 

Confidence Lev 

0.338749 


Q#26 

Q#27 

Q#28 







Mean 

2.266667 

Mean 

1.266667 

Mean 

1.566667 

Standard Error 

0.248829 

Standard Error 

0.11679 

Standard Error 

0.123952 

Median 

2 

Median 

1 

Median 

1.5 

Mode 

1 

Mode 

1 

Mode 

1 

Standard Devia 

1.362891 

Standard Devia 

0.639684 

Standard Devia 

0.678911 

Sample Varianc 

1.857471 

Sample Varianc 

0.409195 

Sample Varianc 

0.46092 

Kurtosis 

-1.77704 

Kurtosis 

3.701688 

Kurtosis 

4.070435 

Skewness 

0.355192 

Skewness 

2.249556 

Skewness 

1.513353 

Range 

3 

Range 

2 

Range 

3 

Minimum 

1 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

3 

Maximum 

4 

Sum 

68 

Sum 

38 

Sum 

47 

Count 

30 

Count 

30 

Count 

30 

Confidence Lev 

0.508912 

Confidence Lev 

0.238862 

Confidence Lev 

0.25351 


Q#29 

Q#30 





Mean 

1.966667 

Mean 

2.166667 

Standard Error 

0.169403 

Standard Error 

0.159621 

Median 

2 

Median 

2 

Mode 

2 

Mode 

3 

Standard Devia 

0.927857 

Standard Devia 

0.874281 

Sample Varianc 

0.86092 

Sample Varianc 

0.764368 

Kurtosis 

0.293101 

Kurtosis 

-1.05533 

Skewness 

0.901792 

Skewness 

-0.01229 

Range 

3 

Range 

3 

Minimum 

1 

Minimum 

1 

Maximum 

4 

Maximum 

4 

Sum 

59 

Sum 

65 

Count 

30 

Count 

30 

Confidence Lev 

0.346468 

Confidence Lev 

0.326462 
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APPENDIX F. LESSONS LEARNED DURING RESEARCH 


Three humbling lessons were learned during the development and analysis of this 
researeh projeet. The first was the ineorreet assumption on my part that the survey partieipants 
would be more eager to partieipate since the research was targeted to justify additional resources 
to facilitate their shortages in information assurance tools, training, funding, etc. The second was 
the assumption that MS Word could maintain very large files. Lastly, the belief that the electric 
company would have an uninterrupted power supply during data compilation was an inaccurate 
assumption. 

Surveys in general are not something that people do to pass the time of day. I found 
myself doing quite a large number of follow up call and emails to encourage participation to an 
acceptable survey sample size. Since the database was anonymously populated, each solicited 
participant had to be contacted since there was no way to determine who had submitted a survey 
response. Anyone attempting to call around the globe should seriously consider purchasing 
prepaid phone cards or invest in a broadband phone to offset phone usage fees. 

Always ensure you know the processing limitation of your software applications. Some 
applications are not forgiving of those that have not determined this in advance. If you intend to 
utilize the NFS thesis template, know that graphics and multiple pages of text add to the total file 
size rather quickly. Documents drafted in MS Word 2002 and MS Word XP docs can be created 
successfully up to 17 MB in file size. However, minor problems begin at around 13 MB if you 
start moving anchors, copying & pasting, TOC, indexing, etc. This issue can induce much 
aggravation and it is much more convenient to configure the maximum file size on Word 
documents to 12 MB. 
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When compiling large amounts of information, be sure that your computer automatically 
saves your information at a minimum of every five minutes. Power outages can occur in Marina, 
California on sunny days in the same way they occur during severe thunder storm days in the 
Midwest. Interruptions in electrical power can promote unnecessary increases in the blood 
pressure and heart rate when your document has not been saved recently...regardless of 
operating system. 
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